Jun 15 09:16:06.160371 ip-10-0-129-143 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 15 09:16:06.160383 ip-10-0-129-143 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 15 09:16:06.160391 ip-10-0-129-143 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 15 09:16:06.160694 ip-10-0-129-143 systemd[1]: Failed to start Kubernetes Kubelet. Jun 15 09:16:16.204269 ip-10-0-129-143 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 15 09:16:16.204286 ip-10-0-129-143 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 1e64bed4108c4c2ab4bfae53ac802353 -- Jun 15 09:18:35.313092 ip-10-0-129-143 systemd[1]: Starting Kubernetes Kubelet... Jun 15 09:18:35.915128 ip-10-0-129-143 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 15 09:18:35.915128 ip-10-0-129-143 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 15 09:18:35.915128 ip-10-0-129-143 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 15 09:18:35.915128 ip-10-0-129-143 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 15 09:18:35.915128 ip-10-0-129-143 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 15 09:18:35.915128 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.768593 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 15 09:18:35.915128 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772656 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 09:18:35.915128 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772668 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 09:18:35.915128 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772671 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 09:18:35.915128 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772675 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 09:18:35.915128 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772680 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 09:18:35.915128 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772684 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 09:18:35.915128 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772687 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 09:18:35.842903 ip-10-0-129-143 systemd[1]: Started Kubernetes Kubelet. Jun 15 09:18:35.931475 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772690 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 09:18:35.931475 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772693 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 09:18:35.931475 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772696 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 09:18:35.931475 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772699 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 09:18:35.931475 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772701 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 15 09:18:35.931475 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772704 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 09:18:35.931475 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772707 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 09:18:35.931475 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772710 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 09:18:35.931475 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772712 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 09:18:35.931475 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772714 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 09:18:35.931475 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772718 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 09:18:35.931475 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772725 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 09:18:35.931475 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772728 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 09:18:35.931475 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772731 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 09:18:35.931475 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772733 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 09:18:35.931475 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772736 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 09:18:35.931475 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772739 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 09:18:35.931475 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772741 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 09:18:35.931475 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772744 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 09:18:35.931475 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772746 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 09:18:35.932187 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772749 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 09:18:35.932187 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772752 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 09:18:35.932187 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772755 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 09:18:35.932187 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772758 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 09:18:35.932187 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772761 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 09:18:35.932187 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772763 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 09:18:35.932187 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772766 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 09:18:35.932187 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772768 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 09:18:35.932187 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772771 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 09:18:35.932187 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772774 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 09:18:35.932187 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772776 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 09:18:35.932187 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772780 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 09:18:35.932187 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772784 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 09:18:35.932187 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772787 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 09:18:35.932187 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772790 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 09:18:35.932187 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772793 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 09:18:35.932187 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772796 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 09:18:35.932187 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772798 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 09:18:35.932187 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772801 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 09:18:35.932991 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772804 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 09:18:35.932991 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772806 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 09:18:35.932991 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772809 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 09:18:35.932991 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772812 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 09:18:35.932991 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772815 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 09:18:35.932991 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772817 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 09:18:35.932991 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772820 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 09:18:35.932991 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772823 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 09:18:35.932991 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772825 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 09:18:35.932991 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772828 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 09:18:35.932991 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772843 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 09:18:35.932991 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772846 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 09:18:35.932991 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772849 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 09:18:35.932991 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772853 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 09:18:35.932991 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772856 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 09:18:35.932991 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772858 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 09:18:35.932991 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772861 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 09:18:35.932991 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772864 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 09:18:35.932991 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772867 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 09:18:35.932991 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772869 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 09:18:35.933564 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772872 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 09:18:35.933564 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772874 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 09:18:35.933564 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772878 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 09:18:35.933564 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772881 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 09:18:35.933564 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772883 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 09:18:35.933564 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772886 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 09:18:35.933564 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772889 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 09:18:35.933564 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772891 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 09:18:35.933564 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772894 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 09:18:35.933564 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772896 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 09:18:35.933564 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772899 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 09:18:35.933564 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772902 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 09:18:35.933564 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772904 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 09:18:35.933564 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772907 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 09:18:35.933564 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772909 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 09:18:35.933564 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772912 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 09:18:35.933564 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772915 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 09:18:35.933564 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772917 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 09:18:35.933564 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772920 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 09:18:35.933564 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772923 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 09:18:35.934275 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772925 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 09:18:35.934275 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772928 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 09:18:35.934275 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772930 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 09:18:35.934275 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772933 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 09:18:35.934275 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772936 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 09:18:35.934275 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772939 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 09:18:35.934275 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772942 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 09:18:35.934275 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772944 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 09:18:35.934275 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772947 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 09:18:35.934275 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772949 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 09:18:35.934275 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772952 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 09:18:35.934275 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772954 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 09:18:35.934275 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.772957 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 09:18:35.934275 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773638 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 09:18:35.934275 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773644 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 09:18:35.934275 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773647 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 09:18:35.934275 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773650 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 09:18:35.934275 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773653 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 09:18:35.934275 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773656 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 09:18:35.934882 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773659 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 09:18:35.934882 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773661 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 09:18:35.934882 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773664 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 09:18:35.934882 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773667 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 09:18:35.934882 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773669 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 15 09:18:35.934882 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773672 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 09:18:35.934882 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773675 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 09:18:35.934882 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773678 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 09:18:35.934882 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773681 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 09:18:35.934882 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773684 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 09:18:35.934882 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773687 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 09:18:35.934882 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773699 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 09:18:35.934882 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773702 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 09:18:35.934882 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773704 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 09:18:35.934882 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773709 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 09:18:35.934882 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773713 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 09:18:35.934882 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773716 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 09:18:35.934882 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773719 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 09:18:35.934882 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773722 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 09:18:35.934882 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773725 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 09:18:35.935541 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773728 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 09:18:35.935541 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773731 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 09:18:35.935541 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773734 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 09:18:35.935541 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773737 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 09:18:35.935541 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773739 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 09:18:35.935541 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773742 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 09:18:35.935541 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773745 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 09:18:35.935541 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773747 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 09:18:35.935541 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773750 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 09:18:35.935541 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773752 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 09:18:35.935541 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773755 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 09:18:35.935541 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773758 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 09:18:35.935541 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773761 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 09:18:35.935541 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773765 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 09:18:35.935541 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773769 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 09:18:35.935541 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773772 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 09:18:35.935541 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773774 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 09:18:35.935541 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773779 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 09:18:35.935541 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773781 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 09:18:35.935541 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773784 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 09:18:35.936271 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773787 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 09:18:35.936271 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773790 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 09:18:35.936271 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773793 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 09:18:35.936271 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773795 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 09:18:35.936271 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773798 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 09:18:35.936271 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773801 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 09:18:35.936271 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773803 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 09:18:35.936271 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773806 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 09:18:35.936271 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773809 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 09:18:35.936271 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773812 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 09:18:35.936271 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773815 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 09:18:35.936271 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773818 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 09:18:35.936271 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773820 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 09:18:35.936271 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773823 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 09:18:35.936271 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773825 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 09:18:35.936271 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773828 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 09:18:35.936271 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773830 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 09:18:35.936271 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773849 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 09:18:35.936271 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773852 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 09:18:35.936271 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773854 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 09:18:35.936866 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773901 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 09:18:35.936866 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773905 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 09:18:35.936866 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773908 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 09:18:35.936866 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773911 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 09:18:35.936866 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773913 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 09:18:35.936866 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773916 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 09:18:35.936866 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773919 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 09:18:35.936866 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773922 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 09:18:35.936866 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773925 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 09:18:35.936866 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773928 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 09:18:35.936866 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773931 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 09:18:35.936866 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773934 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 09:18:35.936866 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773939 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 09:18:35.936866 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773941 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 09:18:35.936866 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773944 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 09:18:35.936866 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773946 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 09:18:35.936866 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773949 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 09:18:35.936866 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773953 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 09:18:35.936866 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773955 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 09:18:35.936866 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773958 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 09:18:35.937415 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773961 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 09:18:35.937415 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773963 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 09:18:35.937415 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773966 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 09:18:35.937415 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773969 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 09:18:35.937415 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773971 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 09:18:35.937415 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773974 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 09:18:35.937415 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773976 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 09:18:35.937415 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773980 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 09:18:35.937415 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773984 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 09:18:35.937415 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773987 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 09:18:35.937415 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773990 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 09:18:35.937415 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773993 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 09:18:35.937415 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.773996 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 09:18:35.937415 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774355 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 09:18:35.937415 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774361 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 09:18:35.937415 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774364 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 09:18:35.937415 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774367 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 09:18:35.937415 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774370 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 15 09:18:35.937415 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774373 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 09:18:35.937415 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774376 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 09:18:35.937963 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774378 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 09:18:35.937963 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774383 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 09:18:35.937963 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774385 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 09:18:35.937963 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774388 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 09:18:35.937963 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774393 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 09:18:35.937963 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774396 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 09:18:35.937963 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774398 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 09:18:35.937963 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774401 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 09:18:35.937963 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774404 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 09:18:35.937963 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774406 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 09:18:35.937963 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774409 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 09:18:35.937963 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774412 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 09:18:35.937963 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774414 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 09:18:35.937963 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774417 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 09:18:35.937963 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774419 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 09:18:35.937963 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774422 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 09:18:35.937963 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774425 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 09:18:35.937963 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774427 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 09:18:35.937963 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774430 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 09:18:35.937963 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774432 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 09:18:35.938514 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774435 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 09:18:35.938514 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774438 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 09:18:35.938514 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774441 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 09:18:35.938514 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774443 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 09:18:35.938514 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774446 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 09:18:35.938514 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774449 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 09:18:35.938514 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774453 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 09:18:35.938514 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774457 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 09:18:35.938514 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774460 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 09:18:35.938514 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774462 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 09:18:35.938514 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774465 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 09:18:35.938514 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774468 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 09:18:35.938514 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774471 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 09:18:35.938514 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774475 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 09:18:35.938514 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774478 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 09:18:35.938514 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774480 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 09:18:35.938514 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774485 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 09:18:35.938514 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774487 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 09:18:35.938514 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774490 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 09:18:35.938986 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774492 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 09:18:35.938986 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774495 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 09:18:35.938986 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774498 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 09:18:35.938986 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774500 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 09:18:35.938986 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774503 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 09:18:35.938986 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774505 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 09:18:35.938986 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774508 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 09:18:35.938986 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774510 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 09:18:35.938986 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774513 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 09:18:35.938986 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774516 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 09:18:35.938986 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774518 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 09:18:35.938986 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774521 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 09:18:35.938986 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774523 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 09:18:35.938986 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774526 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 09:18:35.938986 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774528 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 09:18:35.938986 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774531 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 09:18:35.938986 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774533 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 09:18:35.938986 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774536 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 09:18:35.938986 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774538 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 09:18:35.938986 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774541 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 09:18:35.939464 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774543 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 09:18:35.939464 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774546 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 09:18:35.939464 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774548 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 09:18:35.939464 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774551 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 09:18:35.939464 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774554 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 09:18:35.939464 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774556 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 09:18:35.939464 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774561 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 09:18:35.939464 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774563 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 09:18:35.939464 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774567 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 09:18:35.939464 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774573 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 09:18:35.939464 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774576 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 09:18:35.939464 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774579 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 09:18:35.939464 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774582 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 09:18:35.939464 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774584 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 09:18:35.939464 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774587 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 09:18:35.939464 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774589 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 09:18:35.939464 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774593 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 09:18:35.939464 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774597 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 09:18:35.939464 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774600 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 09:18:35.939943 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774602 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 09:18:35.939943 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774605 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 09:18:35.939943 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774607 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 09:18:35.939943 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774610 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 09:18:35.939943 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774612 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 09:18:35.939943 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774615 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 09:18:35.939943 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774617 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 09:18:35.939943 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774620 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 09:18:35.939943 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774622 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 09:18:35.939943 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774625 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 09:18:35.939943 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774627 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 09:18:35.939943 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774630 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 09:18:35.939943 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774632 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 09:18:35.939943 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.774635 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 09:18:35.939943 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774705 2562 flags.go:64] FLAG: --address="0.0.0.0" Jun 15 09:18:35.939943 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774713 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 15 09:18:35.939943 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774718 2562 flags.go:64] FLAG: --anonymous-auth="true" Jun 15 09:18:35.939943 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774722 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 15 09:18:35.939943 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774727 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 15 09:18:35.939943 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774731 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 15 09:18:35.939943 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774735 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774739 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774743 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774746 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774750 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774753 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774756 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774759 2562 flags.go:64] FLAG: --cgroup-root="" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774762 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774765 2562 flags.go:64] FLAG: --client-ca-file="" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774768 2562 flags.go:64] FLAG: --cloud-provider="external" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774771 2562 flags.go:64] FLAG: --cluster-dns="[]" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774776 2562 flags.go:64] FLAG: --cluster-domain="" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774779 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774782 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774785 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774788 2562 flags.go:64] FLAG: --container-log-max-files="5" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774792 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774795 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774798 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774801 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774804 2562 flags.go:64] FLAG: --contention-profiling="false" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774807 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774810 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 15 09:18:35.940436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774813 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774816 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774820 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774823 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774826 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774828 2562 flags.go:64] FLAG: --enable-load-reader="false" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774846 2562 flags.go:64] FLAG: --enable-server="true" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774851 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774855 2562 flags.go:64] FLAG: --event-burst="100" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774858 2562 flags.go:64] FLAG: --event-qps="50" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774862 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774865 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774868 2562 flags.go:64] FLAG: --eviction-hard="" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774872 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774875 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774878 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774881 2562 flags.go:64] FLAG: --eviction-soft="" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774884 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774886 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774889 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774892 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774895 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774898 2562 flags.go:64] FLAG: --fail-swap-on="true" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774901 2562 flags.go:64] FLAG: --feature-gates="" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774905 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jun 15 09:18:35.941021 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774908 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774911 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774914 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774917 2562 flags.go:64] FLAG: --healthz-port="10248" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774920 2562 flags.go:64] FLAG: --help="false" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774923 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-129-143.ec2.internal" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774925 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774928 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774931 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774934 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774938 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774940 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774943 2562 flags.go:64] FLAG: --image-service-endpoint="" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774946 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774950 2562 flags.go:64] FLAG: --kube-api-burst="100" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774953 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774956 2562 flags.go:64] FLAG: --kube-api-qps="50" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774961 2562 flags.go:64] FLAG: --kube-reserved="" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774963 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774966 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774969 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774972 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774975 2562 flags.go:64] FLAG: --lock-file="" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774978 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 15 09:18:35.941667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774981 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 15 09:18:35.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774984 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 15 09:18:35.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774989 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jun 15 09:18:35.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774992 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 15 09:18:35.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774994 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jun 15 09:18:35.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.774997 2562 flags.go:64] FLAG: --logging-format="text" Jun 15 09:18:35.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775000 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 15 09:18:35.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775003 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 15 09:18:35.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775006 2562 flags.go:64] FLAG: --manifest-url="" Jun 15 09:18:35.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775009 2562 flags.go:64] FLAG: --manifest-url-header="" Jun 15 09:18:35.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775013 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 15 09:18:35.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775016 2562 flags.go:64] FLAG: --max-open-files="1000000" Jun 15 09:18:35.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775020 2562 flags.go:64] FLAG: --max-pods="110" Jun 15 09:18:35.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775023 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 15 09:18:35.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775026 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 15 09:18:35.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775029 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jun 15 09:18:35.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775031 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 15 09:18:35.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775034 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 15 09:18:35.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775037 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 15 09:18:35.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775040 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 15 09:18:35.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775047 2562 flags.go:64] FLAG: --node-status-max-images="50" Jun 15 09:18:35.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775050 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 15 09:18:35.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775054 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jun 15 09:18:35.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775058 2562 flags.go:64] FLAG: --pod-cidr="" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775061 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4a2e90c8ef7d783e4d635e24fb09435f5e1f6e954583512c0944f9e57bc77b52" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775068 2562 flags.go:64] FLAG: --pod-manifest-path="" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775071 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775074 2562 flags.go:64] FLAG: --pods-per-core="0" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775077 2562 flags.go:64] FLAG: --port="10250" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775080 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775083 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-039fc50982db5db7b" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775086 2562 flags.go:64] FLAG: --qos-reserved="" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775089 2562 flags.go:64] FLAG: --read-only-port="10255" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775092 2562 flags.go:64] FLAG: --register-node="true" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775095 2562 flags.go:64] FLAG: --register-with-taints="" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775098 2562 flags.go:64] FLAG: --registry-burst="10" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775101 2562 flags.go:64] FLAG: --registry-qps="5" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775104 2562 flags.go:64] FLAG: --reserved-cpus="" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775107 2562 flags.go:64] FLAG: --reserved-memory="" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775110 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775113 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775116 2562 flags.go:64] FLAG: --rotate-certificates="false" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775119 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775122 2562 flags.go:64] FLAG: --runonce="false" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775125 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775129 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775131 2562 flags.go:64] FLAG: --seccomp-default="false" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775134 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 15 09:18:35.942906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775137 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775140 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775143 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775158 2562 flags.go:64] FLAG: --storage-driver-password="root" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775162 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775166 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775169 2562 flags.go:64] FLAG: --storage-driver-user="root" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775173 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775176 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775181 2562 flags.go:64] FLAG: --system-cgroups="" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775184 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775189 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775192 2562 flags.go:64] FLAG: --tls-cert-file="" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775195 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775199 2562 flags.go:64] FLAG: --tls-min-version="" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775202 2562 flags.go:64] FLAG: --tls-private-key-file="" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775205 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775207 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775210 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775213 2562 flags.go:64] FLAG: --v="2" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775217 2562 flags.go:64] FLAG: --version="false" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775223 2562 flags.go:64] FLAG: --vmodule="" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775227 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.775230 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775339 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 09:18:35.943504 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775342 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 09:18:35.944134 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775345 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 09:18:35.944134 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775348 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 09:18:35.944134 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775351 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 09:18:35.944134 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775354 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 09:18:35.944134 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775356 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 09:18:35.944134 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775359 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 09:18:35.944134 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775362 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 09:18:35.944134 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775364 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 09:18:35.944134 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775367 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 09:18:35.944134 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775369 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 09:18:35.944134 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775372 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 09:18:35.944134 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775374 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 09:18:35.944134 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775377 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 09:18:35.944134 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775382 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 09:18:35.944134 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775384 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 09:18:35.944134 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775389 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 09:18:35.944134 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775392 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 09:18:35.944134 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775394 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 09:18:35.944134 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775397 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 09:18:35.944134 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775399 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 09:18:35.944653 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775402 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 09:18:35.944653 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775405 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 09:18:35.944653 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775407 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 09:18:35.944653 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775410 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 09:18:35.944653 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775412 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 09:18:35.944653 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775415 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 15 09:18:35.944653 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775418 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 09:18:35.944653 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775420 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 09:18:35.944653 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775423 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 09:18:35.944653 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775426 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 09:18:35.944653 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775428 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 09:18:35.944653 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775431 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 09:18:35.944653 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775433 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 09:18:35.944653 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775437 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 09:18:35.944653 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775441 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 09:18:35.944653 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775444 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 09:18:35.944653 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775447 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 09:18:35.944653 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775449 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 09:18:35.944653 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775452 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 09:18:35.944653 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775455 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 09:18:35.945154 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775457 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 09:18:35.945154 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775460 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 09:18:35.945154 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775463 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 09:18:35.945154 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775466 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 09:18:35.945154 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775469 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 09:18:35.945154 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775473 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 09:18:35.945154 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775476 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 09:18:35.945154 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775479 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 09:18:35.945154 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775482 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 09:18:35.945154 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775485 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 09:18:35.945154 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775488 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 09:18:35.945154 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775491 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 09:18:35.945154 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775493 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 09:18:35.945154 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775496 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 09:18:35.945154 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775499 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 09:18:35.945154 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775501 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 09:18:35.945154 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775504 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 09:18:35.945154 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775506 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 09:18:35.945154 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775509 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 09:18:35.945154 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775512 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 09:18:35.945641 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775514 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 09:18:35.945641 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775517 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 09:18:35.945641 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775519 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 09:18:35.945641 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775522 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 09:18:35.945641 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775525 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 09:18:35.945641 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775527 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 09:18:35.945641 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775530 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 09:18:35.945641 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775533 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 09:18:35.945641 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775535 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 09:18:35.945641 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775538 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 09:18:35.945641 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775541 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 09:18:35.945641 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775543 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 09:18:35.945641 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775546 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 09:18:35.945641 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775548 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 09:18:35.945641 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775551 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 09:18:35.945641 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775553 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 09:18:35.945641 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775556 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 09:18:35.945641 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775560 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 09:18:35.946110 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775564 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 09:18:35.946110 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775568 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 09:18:35.946110 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775572 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 09:18:35.946110 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775575 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 09:18:35.946110 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775578 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 09:18:35.946110 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775581 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 09:18:35.946110 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775583 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 09:18:35.946110 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775586 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 09:18:35.946110 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775589 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 09:18:35.946110 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775591 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 09:18:35.946110 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775594 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 09:18:35.946110 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775597 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 09:18:35.946110 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775599 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 09:18:35.946110 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775603 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 09:18:35.946110 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775607 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 09:18:35.946110 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775609 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 09:18:35.946110 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775612 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 09:18:35.946110 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775614 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 09:18:35.946110 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.775617 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 09:18:35.946572 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.776475 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 15 09:18:35.946572 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.785359 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 15 09:18:35.946572 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.785375 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 15 09:18:35.946572 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.785391 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 15 09:18:35.946572 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.785400 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 15 09:18:35.946572 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785459 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 09:18:35.946572 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785463 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 09:18:35.946572 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785467 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 09:18:35.946572 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785469 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 09:18:35.946572 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785472 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 09:18:35.946572 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785475 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 09:18:35.946572 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785479 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 09:18:35.946572 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785483 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 09:18:35.946572 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785486 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 09:18:35.946946 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785489 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 09:18:35.946946 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785492 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 09:18:35.946946 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785495 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 09:18:35.946946 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785498 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 09:18:35.946946 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785500 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 09:18:35.946946 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785503 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 09:18:35.946946 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785506 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 09:18:35.946946 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785509 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 09:18:35.946946 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785511 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 09:18:35.946946 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785514 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 09:18:35.946946 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785517 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 09:18:35.946946 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785520 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 09:18:35.946946 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785522 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 09:18:35.946946 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785525 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 09:18:35.946946 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785531 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 09:18:35.946946 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785534 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 09:18:35.946946 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785537 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 09:18:35.946946 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785540 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 09:18:35.946946 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785543 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 09:18:35.946946 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785546 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 09:18:35.947462 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785554 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 09:18:35.947462 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785558 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 09:18:35.947462 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785562 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 09:18:35.947462 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785564 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 09:18:35.947462 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785568 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 09:18:35.947462 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785572 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 09:18:35.947462 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785575 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 09:18:35.947462 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785578 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 09:18:35.947462 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785581 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 09:18:35.947462 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785584 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 09:18:35.947462 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785587 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 09:18:35.947462 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785590 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 09:18:35.947462 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785593 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 09:18:35.947462 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785596 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 09:18:35.947462 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785599 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 09:18:35.947462 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785601 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 09:18:35.947462 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785604 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 15 09:18:35.947462 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785606 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 09:18:35.947462 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785609 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 09:18:35.947972 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785611 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 09:18:35.947972 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785614 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 09:18:35.947972 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785616 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 09:18:35.947972 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785619 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 09:18:35.947972 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785621 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 09:18:35.947972 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785624 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 09:18:35.947972 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785626 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 09:18:35.947972 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785631 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 09:18:35.947972 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785634 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 09:18:35.947972 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785637 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 09:18:35.947972 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785639 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 09:18:35.947972 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785642 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 09:18:35.947972 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785644 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 09:18:35.947972 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785647 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 09:18:35.947972 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785655 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 09:18:35.947972 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785657 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 09:18:35.947972 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785660 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 09:18:35.947972 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785663 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 09:18:35.947972 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785665 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 09:18:35.947972 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785668 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 09:18:35.948441 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785670 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 09:18:35.948441 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785673 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 09:18:35.948441 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785675 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 09:18:35.948441 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785678 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 09:18:35.948441 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785680 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 09:18:35.948441 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785683 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 09:18:35.948441 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785686 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 09:18:35.948441 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785695 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 09:18:35.948441 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785699 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 09:18:35.948441 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785702 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 09:18:35.948441 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785705 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 09:18:35.948441 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785708 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 09:18:35.948441 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785710 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 09:18:35.948441 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785713 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 09:18:35.948441 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785715 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 09:18:35.948441 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785718 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 09:18:35.948441 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785720 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 09:18:35.948441 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785723 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 09:18:35.948441 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785725 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 09:18:35.948441 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785729 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 09:18:35.948930 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785732 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 09:18:35.948930 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785735 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 09:18:35.948930 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785737 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 09:18:35.948930 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785740 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 09:18:35.948930 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785743 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 09:18:35.948930 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785746 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 09:18:35.948930 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785749 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 09:18:35.948930 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785759 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 09:18:35.948930 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785761 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 09:18:35.948930 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785764 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 09:18:35.948930 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785767 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 09:18:35.948930 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.785772 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 15 09:18:35.948930 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785931 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 09:18:35.948930 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785937 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 09:18:35.948930 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785940 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 09:18:35.949289 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785943 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 09:18:35.949289 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785946 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 09:18:35.949289 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785948 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 09:18:35.949289 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785951 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 09:18:35.949289 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785953 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 09:18:35.949289 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785956 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 09:18:35.949289 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785958 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 15 09:18:35.949289 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785961 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 09:18:35.949289 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785964 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 09:18:35.949289 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785966 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 09:18:35.949289 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785969 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 09:18:35.949289 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785971 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 09:18:35.949289 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785974 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 09:18:35.949289 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785977 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 09:18:35.949289 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785979 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 09:18:35.949289 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785982 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 09:18:35.949289 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785984 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 09:18:35.949289 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785991 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 09:18:35.949289 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785993 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 09:18:35.949752 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.785997 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 09:18:35.949752 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786000 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 09:18:35.949752 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786003 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 09:18:35.949752 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786006 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 09:18:35.949752 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786008 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 09:18:35.949752 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786011 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 09:18:35.949752 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786019 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 09:18:35.949752 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786022 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 09:18:35.949752 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786024 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 09:18:35.949752 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786027 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 09:18:35.949752 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786029 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 09:18:35.949752 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786032 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 09:18:35.949752 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786034 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 09:18:35.949752 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786037 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 09:18:35.949752 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786039 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 09:18:35.949752 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786042 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 09:18:35.949752 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786045 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 09:18:35.949752 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786047 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 09:18:35.949752 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786050 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 09:18:35.950224 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786052 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 09:18:35.950224 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786055 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 09:18:35.950224 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786057 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 09:18:35.950224 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786060 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 09:18:35.950224 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786062 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 09:18:35.950224 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786065 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 09:18:35.950224 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786067 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 09:18:35.950224 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786070 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 09:18:35.950224 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786072 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 09:18:35.950224 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786074 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 09:18:35.950224 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786077 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 09:18:35.950224 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786082 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 09:18:35.950224 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786084 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 09:18:35.950224 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786087 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 09:18:35.950224 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786089 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 09:18:35.950224 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786092 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 09:18:35.950224 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786095 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 09:18:35.950224 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786097 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 09:18:35.950224 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786099 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 09:18:35.950224 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786102 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 09:18:35.950688 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786110 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 09:18:35.950688 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786113 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 09:18:35.950688 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786115 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 09:18:35.950688 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786118 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 09:18:35.950688 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786120 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 09:18:35.950688 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786125 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 09:18:35.950688 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786128 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 09:18:35.950688 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786131 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 09:18:35.950688 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786135 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 09:18:35.950688 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786138 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 09:18:35.950688 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786141 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 09:18:35.950688 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786144 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 09:18:35.950688 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786147 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 09:18:35.950688 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786150 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 09:18:35.950688 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786152 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 09:18:35.950688 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786155 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 09:18:35.950688 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786158 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 09:18:35.950688 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786160 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 09:18:35.950688 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786163 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 09:18:35.951180 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786165 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 09:18:35.951180 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786167 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 09:18:35.951180 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786170 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 09:18:35.951180 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786173 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 09:18:35.951180 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786180 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 09:18:35.951180 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786183 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 09:18:35.951180 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786185 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 09:18:35.951180 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786188 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 09:18:35.951180 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786191 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 09:18:35.951180 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786193 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 09:18:35.951180 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786196 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 09:18:35.951180 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786199 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 09:18:35.951180 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786201 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 09:18:35.951180 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786204 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 09:18:35.951180 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786212 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 09:18:35.951180 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786215 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 09:18:35.951180 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786218 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 09:18:35.951180 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786220 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 09:18:35.951180 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:35.786223 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 09:18:35.951628 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.786227 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 15 09:18:35.951628 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.786354 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jun 15 09:18:35.951628 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.791179 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 15 09:18:35.951628 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.792455 2562 server.go:1019] "Starting client certificate rotation" Jun 15 09:18:35.951628 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.792556 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 15 09:18:35.951628 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.792589 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 15 09:18:35.951628 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.795591 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 15 09:18:35.951628 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.796814 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 15 09:18:35.951628 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.809496 2562 log.go:25] "Validated CRI v1 runtime API" Jun 15 09:18:35.951628 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.813067 2562 log.go:25] "Validated CRI v1 image API" Jun 15 09:18:35.951628 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.813822 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 15 09:18:35.951628 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.816470 2562 fs.go:135] Filesystem UUIDs: map[744c3013-56aa-40f7-9f2e-baa46fb19cd0:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 ed65c7a8-2aee-4ea2-bb2b-b21f0fbaf353:/dev/nvme0n1p4] Jun 15 09:18:35.952053 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.816495 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 15 09:18:35.952089 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.823449 2562 manager.go:216] Machine: {Timestamp:2026-06-15 09:18:35.821941438 +0000 UTC m=+0.385363742 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102133 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec223a6c6ec48a5f95770b95c3afabe6 SystemUUID:ec223a6c-6ec4-8a5f-9577-0b95c3afabe6 BootID:1e64bed4-108c-4c2a-b4bf-ae53ac802353 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c0:0a:45:71:07 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c0:0a:45:71:07 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:dd:aa:a7:34:2a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 15 09:18:35.952089 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.823581 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 15 09:18:35.952089 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.823666 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.120.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260608-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 15 09:18:35.952089 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.826414 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 15 09:18:35.952089 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.826441 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-143.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 15 09:18:35.952089 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.827074 2562 topology_manager.go:138] "Creating topology manager with none policy" Jun 15 09:18:35.952089 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.827083 2562 container_manager_linux.go:309] "Creating device plugin manager" Jun 15 09:18:35.952089 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.827090 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.827856 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.828995 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.831947 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.832315 2562 state_mem.go:36] "Initialized new in-memory state store" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.832453 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.832483 2562 kubelet.go:486] "Attempting to sync node with API server" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.832504 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.832563 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.832611 2562 kubelet.go:398] "Adding apiserver pod source" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.832629 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.833058 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.833075 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.836418 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.838332 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.838361 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.838922 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.838935 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.838941 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.838947 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.838959 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.838968 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.839227 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.839238 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.839246 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.839252 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.839261 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.839272 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.839297 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.839303 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.841313 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-143.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:35.841534 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-143.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:35.841677 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.842180 2562 server.go:1268] "Started kubelet" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.842296 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ghd8s" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.842310 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.842359 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.842425 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.843923 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 15 09:18:35.952324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.844777 2562 server.go:310] "Adding debug handlers to kubelet server" Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:35.849051 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:35.848588 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-143.ec2.internal.18b9368f19c2a95b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-143.ec2.internal,UID:ip-10-0-129-143.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-143.ec2.internal,},FirstTimestamp:2026-06-15 09:18:35.842152795 +0000 UTC m=+0.405575094,LastTimestamp:2026-06-15 09:18:35.842152795 +0000 UTC m=+0.405575094,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-143.ec2.internal,}" Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.849784 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ghd8s" Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.849816 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.849801 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:35.850101 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-143.ec2.internal\" not found" Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.850782 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.850814 2562 factory.go:55] Registering systemd factory Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.850824 2562 factory.go:223] Registration of the systemd container factory successfully Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.851086 2562 factory.go:153] Registering CRI-O factory Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.851102 2562 factory.go:223] Registration of the crio container factory successfully Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.851126 2562 factory.go:103] Registering Raw factory Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.851141 2562 manager.go:1196] Started watching for new ooms in manager Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.851365 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.851367 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.851389 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.851505 2562 reconstruct.go:97] "Volume reconstruction finished" Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.851516 2562 reconciler.go:29] "Reconciler: start to sync state" Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.851770 2562 manager.go:349] Starting recovery of all containers Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:35.851815 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:35.851942 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-143.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.865255 2562 manager.go:354] Recovery completed Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.869438 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.871129 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-143.ec2.internal" event="NodeHasSufficientMemory" Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.871158 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-143.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.871170 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-143.ec2.internal" event="NodeHasSufficientPID" Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.871635 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.871645 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.871646 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.871686 2562 state_mem.go:36] "Initialized new in-memory state store" Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.872260 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-143.ec2.internal" event="NodeHasSufficientMemory" Jun 15 09:18:35.953299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.872285 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-143.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 09:18:35.954099 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.872297 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-143.ec2.internal" event="NodeHasSufficientPID" Jun 15 09:18:35.954099 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.872700 2562 policy_none.go:49] "None policy: Start" Jun 15 09:18:35.954099 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.872712 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 15 09:18:35.954099 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.872722 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 15 09:18:35.954099 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.873563 2562 policy_none.go:47] "Start" Jun 15 09:18:35.954099 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.919422 2562 manager.go:337] "Starting Device Plugin manager" Jun 15 09:18:35.954099 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:35.919579 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 15 09:18:35.954099 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.919593 2562 server.go:87] "Starting device plugin registration server" Jun 15 09:18:35.954099 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.919789 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 15 09:18:35.954099 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.919798 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 15 09:18:35.954099 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.919904 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 15 09:18:35.954099 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.919984 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 15 09:18:35.954099 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.919993 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 15 09:18:35.954099 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:35.920502 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 15 09:18:35.954099 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:35.920539 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-143.ec2.internal\" not found" Jun 15 09:18:35.954099 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.926591 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 15 09:18:35.954099 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.929020 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 15 09:18:35.954099 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.929053 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 15 09:18:35.954099 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.929069 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jun 15 09:18:35.954099 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:35.929103 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 15 09:18:35.954099 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:35.930959 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 09:18:36.020121 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.020088 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 09:18:36.021246 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.021230 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-143.ec2.internal" event="NodeHasSufficientMemory" Jun 15 09:18:36.021324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.021263 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-143.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 09:18:36.021324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.021278 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-143.ec2.internal" event="NodeHasSufficientPID" Jun 15 09:18:36.021324 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.021309 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-143.ec2.internal" Jun 15 09:18:36.027588 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.027572 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-143.ec2.internal" Jun 15 09:18:36.027659 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:36.027594 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-143.ec2.internal\": node \"ip-10-0-129-143.ec2.internal\" not found" Jun 15 09:18:36.029718 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.029701 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-143.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-143.ec2.internal"] Jun 15 09:18:36.029772 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.029764 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 09:18:36.030585 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.030564 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-143.ec2.internal" event="NodeHasSufficientMemory" Jun 15 09:18:36.030636 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.030588 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-143.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 09:18:36.030636 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.030603 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-143.ec2.internal" event="NodeHasSufficientPID" Jun 15 09:18:36.032939 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.032926 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 09:18:36.033069 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.033056 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-143.ec2.internal" Jun 15 09:18:36.033103 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.033083 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 09:18:36.033738 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.033724 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-143.ec2.internal" event="NodeHasSufficientMemory" Jun 15 09:18:36.033798 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.033745 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-143.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 09:18:36.033798 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.033729 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-143.ec2.internal" event="NodeHasSufficientMemory" Jun 15 09:18:36.033798 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.033779 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-143.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 09:18:36.033798 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.033792 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-143.ec2.internal" event="NodeHasSufficientPID" Jun 15 09:18:36.033798 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.033755 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-143.ec2.internal" event="NodeHasSufficientPID" Jun 15 09:18:36.035961 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.035946 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-143.ec2.internal" Jun 15 09:18:36.036010 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.035977 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 09:18:36.036634 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.036621 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-143.ec2.internal" event="NodeHasSufficientMemory" Jun 15 09:18:36.036695 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.036641 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-143.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 09:18:36.036695 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.036652 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-143.ec2.internal" event="NodeHasSufficientPID" Jun 15 09:18:36.042098 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:36.042084 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-143.ec2.internal\" not found" Jun 15 09:18:36.052483 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.052467 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5f6ea51b0527deb3dc5845d11ce6220c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-143.ec2.internal\" (UID: \"5f6ea51b0527deb3dc5845d11ce6220c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-143.ec2.internal" Jun 15 09:18:36.052537 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.052488 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f6ea51b0527deb3dc5845d11ce6220c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-143.ec2.internal\" (UID: \"5f6ea51b0527deb3dc5845d11ce6220c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-143.ec2.internal" Jun 15 09:18:36.052537 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.052508 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6546904cab304243cc28b9f8f6f2abcd-config\") pod \"kube-apiserver-proxy-ip-10-0-129-143.ec2.internal\" (UID: \"6546904cab304243cc28b9f8f6f2abcd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-143.ec2.internal" Jun 15 09:18:36.066491 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:36.066469 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-143.ec2.internal\" not found" node="ip-10-0-129-143.ec2.internal" Jun 15 09:18:36.070721 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:36.070705 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-143.ec2.internal\" not found" node="ip-10-0-129-143.ec2.internal" Jun 15 09:18:36.142399 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:36.142384 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-143.ec2.internal\" not found" Jun 15 09:18:36.153039 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.153023 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5f6ea51b0527deb3dc5845d11ce6220c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-143.ec2.internal\" (UID: \"5f6ea51b0527deb3dc5845d11ce6220c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-143.ec2.internal" Jun 15 09:18:36.153095 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.153047 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f6ea51b0527deb3dc5845d11ce6220c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-143.ec2.internal\" (UID: \"5f6ea51b0527deb3dc5845d11ce6220c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-143.ec2.internal" Jun 15 09:18:36.153095 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.153062 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6546904cab304243cc28b9f8f6f2abcd-config\") pod \"kube-apiserver-proxy-ip-10-0-129-143.ec2.internal\" (UID: \"6546904cab304243cc28b9f8f6f2abcd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-143.ec2.internal" Jun 15 09:18:36.153095 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.153085 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6546904cab304243cc28b9f8f6f2abcd-config\") pod \"kube-apiserver-proxy-ip-10-0-129-143.ec2.internal\" (UID: \"6546904cab304243cc28b9f8f6f2abcd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-143.ec2.internal" Jun 15 09:18:36.153178 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.153119 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5f6ea51b0527deb3dc5845d11ce6220c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-143.ec2.internal\" (UID: \"5f6ea51b0527deb3dc5845d11ce6220c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-143.ec2.internal" Jun 15 09:18:36.153178 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.153123 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f6ea51b0527deb3dc5845d11ce6220c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-143.ec2.internal\" (UID: \"5f6ea51b0527deb3dc5845d11ce6220c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-143.ec2.internal" Jun 15 09:18:36.242901 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:36.242828 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-143.ec2.internal\" not found" Jun 15 09:18:36.343611 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:36.343583 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-143.ec2.internal\" not found" Jun 15 09:18:36.369102 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.369084 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-143.ec2.internal" Jun 15 09:18:36.373642 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.373628 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-143.ec2.internal" Jun 15 09:18:36.444714 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:36.444682 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-143.ec2.internal\" not found" Jun 15 09:18:36.545237 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:36.545188 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-143.ec2.internal\" not found" Jun 15 09:18:36.645768 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:36.645740 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-143.ec2.internal\" not found" Jun 15 09:18:36.746309 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:36.746288 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-143.ec2.internal\" not found" Jun 15 09:18:36.791849 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.791817 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 15 09:18:36.792008 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.791990 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 15 09:18:36.846462 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:36.846436 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-143.ec2.internal\" not found" Jun 15 09:18:36.853421 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.853384 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-14 09:13:35 +0000 UTC" deadline="2027-12-25 04:45:03.848574798 +0000 UTC" Jun 15 09:18:36.853503 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.853421 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13387h26m26.995158215s" Jun 15 09:18:36.946117 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:36.946077 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f6ea51b0527deb3dc5845d11ce6220c.slice/crio-ac08d224661ae06a6833acb9cada4d104739e7378a7c8185ecafca56a15d61a9 WatchSource:0}: Error finding container ac08d224661ae06a6833acb9cada4d104739e7378a7c8185ecafca56a15d61a9: Status 404 returned error can't find the container with id ac08d224661ae06a6833acb9cada4d104739e7378a7c8185ecafca56a15d61a9 Jun 15 09:18:36.947677 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:36.947287 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6546904cab304243cc28b9f8f6f2abcd.slice/crio-e2975b1d0eab747d1e956fffcc282445d18ef23e0eea4418f7ce1f81d27d8430 WatchSource:0}: Error finding container e2975b1d0eab747d1e956fffcc282445d18ef23e0eea4418f7ce1f81d27d8430: Status 404 returned error can't find the container with id e2975b1d0eab747d1e956fffcc282445d18ef23e0eea4418f7ce1f81d27d8430 Jun 15 09:18:36.947677 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:36.947333 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-143.ec2.internal\" not found" Jun 15 09:18:36.953630 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:36.953614 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 15 09:18:37.019647 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.019622 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 09:18:37.047675 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:37.047656 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-143.ec2.internal\" not found" Jun 15 09:18:37.148228 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:37.148192 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-143.ec2.internal\" not found" Jun 15 09:18:37.248770 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:37.248747 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-143.ec2.internal\" not found" Jun 15 09:18:37.322904 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.322881 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 09:18:37.335688 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.335630 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 09:18:37.350522 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.350340 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-143.ec2.internal" Jun 15 09:18:37.363653 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.363630 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 15 09:18:37.364666 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.364648 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-143.ec2.internal" Jun 15 09:18:37.372178 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.372164 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 15 09:18:37.615418 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.615187 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 09:18:37.833403 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.833374 2562 apiserver.go:52] "Watching apiserver" Jun 15 09:18:37.840671 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.840648 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 15 09:18:37.841148 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.841122 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-143.ec2.internal","openshift-multus/multus-sxdtv","openshift-multus/network-metrics-daemon-7trxt","openshift-network-operator/iptables-alerter-p6nnr","openshift-ovn-kubernetes/ovnkube-node-pxdxj","kube-system/konnectivity-agent-47q6n","kube-system/kube-apiserver-proxy-ip-10-0-129-143.ec2.internal","openshift-dns/node-resolver-jlhbm","openshift-image-registry/node-ca-lfzxm","openshift-multus/multus-additional-cni-plugins-44gsl","openshift-network-diagnostics/network-check-target-cdmf7","kube-system/global-pull-secret-syncer-kddf5","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq","openshift-cluster-node-tuning-operator/tuned-bkgdl"] Jun 15 09:18:37.843666 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.843643 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.845921 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.845898 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" Jun 15 09:18:37.846194 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.846174 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 15 09:18:37.846493 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.846400 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-88trz\"" Jun 15 09:18:37.846493 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.846449 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 15 09:18:37.846651 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.846507 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 15 09:18:37.846651 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.846601 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 15 09:18:37.846753 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.846666 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 15 09:18:37.848114 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.848092 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 15 09:18:37.848248 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.848231 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:18:37.848344 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:37.848319 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7trxt" podUID="67c71a8c-71f7-4494-a8cd-fd7b2af8ab51" Jun 15 09:18:37.848401 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.848095 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pz2vn\"" Jun 15 09:18:37.848510 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.848494 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 15 09:18:37.848776 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.848760 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 15 09:18:37.850532 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.850427 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 15 09:18:37.850631 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.850554 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p6nnr" Jun 15 09:18:37.852693 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.852672 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 15 09:18:37.852788 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.852709 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 15 09:18:37.852867 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.852803 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.853011 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.852995 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rpvlf\"" Jun 15 09:18:37.853224 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.853207 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 15 09:18:37.854752 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.854734 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 15 09:18:37.855053 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.855034 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-47q6n" Jun 15 09:18:37.855383 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.855369 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 15 09:18:37.855498 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.855479 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 15 09:18:37.855717 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.855699 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 15 09:18:37.856506 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.856489 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rwgxs\"" Jun 15 09:18:37.856593 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.856515 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 15 09:18:37.856744 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.856727 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 15 09:18:37.857267 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.857249 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 15 09:18:37.857478 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.857457 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 15 09:18:37.857565 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.857457 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-m5lr9\"" Jun 15 09:18:37.858029 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.858012 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lfzxm" Jun 15 09:18:37.860334 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.860314 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 15 09:18:37.860421 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.860362 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 15 09:18:37.860597 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.860577 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 15 09:18:37.860683 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.860630 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:37.861912 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.861092 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mtm9n\"" Jun 15 09:18:37.861912 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.861555 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 15 09:18:37.862638 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.862245 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w8ks\" (UniqueName: \"kubernetes.io/projected/92f911d1-972a-4024-9cd7-311529a59fb7-kube-api-access-9w8ks\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.862638 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.862287 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-host-kubelet\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.862638 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.862412 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-node-log\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.862638 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.862448 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-host-cni-bin\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.862638 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.862478 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92f911d1-972a-4024-9cd7-311529a59fb7-cnibin\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.862638 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.862524 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/92f911d1-972a-4024-9cd7-311529a59fb7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.862638 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.862581 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d7a4b513-5f5a-49ca-b112-6c4c00223f04-iptables-alerter-script\") pod \"iptables-alerter-p6nnr\" (UID: \"d7a4b513-5f5a-49ca-b112-6c4c00223f04\") " pod="openshift-network-operator/iptables-alerter-p6nnr" Jun 15 09:18:37.862638 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.862611 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-var-lib-openvswitch\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.863066 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.862646 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-host-run-ovn-kubernetes\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.863066 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.862675 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f344540f-2677-49dd-8a0f-106be00f414d-ovnkube-config\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.863066 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.862706 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z4tq\" (UniqueName: \"kubernetes.io/projected/facd9737-c7d9-4fa5-9ece-f598f2c0cf1f-kube-api-access-6z4tq\") pod \"aws-ebs-csi-driver-node-k58cq\" (UID: \"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" Jun 15 09:18:37.863066 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.862737 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqnwc\" (UniqueName: \"kubernetes.io/projected/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-kube-api-access-vqnwc\") pod \"network-metrics-daemon-7trxt\" (UID: \"67c71a8c-71f7-4494-a8cd-fd7b2af8ab51\") " pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:18:37.863066 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.862762 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g8h6\" (UniqueName: \"kubernetes.io/projected/d7a4b513-5f5a-49ca-b112-6c4c00223f04-kube-api-access-2g8h6\") pod \"iptables-alerter-p6nnr\" (UID: \"d7a4b513-5f5a-49ca-b112-6c4c00223f04\") " pod="openshift-network-operator/iptables-alerter-p6nnr" Jun 15 09:18:37.863066 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.862819 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-host-run-netns\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.863066 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.862882 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6dzb\" (UniqueName: \"kubernetes.io/projected/f344540f-2677-49dd-8a0f-106be00f414d-kube-api-access-f6dzb\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.863066 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.862953 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sxdtv" Jun 15 09:18:37.863066 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.862979 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/92f911d1-972a-4024-9cd7-311529a59fb7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.863066 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.863029 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/facd9737-c7d9-4fa5-9ece-f598f2c0cf1f-socket-dir\") pod \"aws-ebs-csi-driver-node-k58cq\" (UID: \"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" Jun 15 09:18:37.863066 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.863058 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/facd9737-c7d9-4fa5-9ece-f598f2c0cf1f-registration-dir\") pod \"aws-ebs-csi-driver-node-k58cq\" (UID: \"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" Jun 15 09:18:37.863590 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.863189 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-etc-openvswitch\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.863590 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.863278 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-run-openvswitch\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.863590 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.863309 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f344540f-2677-49dd-8a0f-106be00f414d-ovn-node-metrics-cert\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.863590 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.863340 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92f911d1-972a-4024-9cd7-311529a59fb7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.863590 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.863544 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/facd9737-c7d9-4fa5-9ece-f598f2c0cf1f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k58cq\" (UID: \"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" Jun 15 09:18:37.863807 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.863578 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/facd9737-c7d9-4fa5-9ece-f598f2c0cf1f-device-dir\") pod \"aws-ebs-csi-driver-node-k58cq\" (UID: \"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" Jun 15 09:18:37.863807 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.863694 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/facd9737-c7d9-4fa5-9ece-f598f2c0cf1f-sys-fs\") pod \"aws-ebs-csi-driver-node-k58cq\" (UID: \"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" Jun 15 09:18:37.863807 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.863722 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs\") pod \"network-metrics-daemon-7trxt\" (UID: \"67c71a8c-71f7-4494-a8cd-fd7b2af8ab51\") " pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:18:37.863807 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.863750 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7a4b513-5f5a-49ca-b112-6c4c00223f04-host-slash\") pod \"iptables-alerter-p6nnr\" (UID: \"d7a4b513-5f5a-49ca-b112-6c4c00223f04\") " pod="openshift-network-operator/iptables-alerter-p6nnr" Jun 15 09:18:37.863807 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.863778 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-run-systemd\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.864062 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.863815 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-host-cni-netd\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.864062 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.863868 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/facd9737-c7d9-4fa5-9ece-f598f2c0cf1f-etc-selinux\") pod \"aws-ebs-csi-driver-node-k58cq\" (UID: \"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" Jun 15 09:18:37.864062 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.863891 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 15 09:18:37.864062 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.863945 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-54x8s\"" Jun 15 09:18:37.864237 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.864202 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-systemd-units\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.864785 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.864280 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-log-socket\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.864785 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.864379 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f344540f-2677-49dd-8a0f-106be00f414d-ovnkube-script-lib\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.864785 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.864449 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92f911d1-972a-4024-9cd7-311529a59fb7-cni-binary-copy\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.864785 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.864545 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-host-slash\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.864785 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.864595 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-run-ovn\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.864785 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.864644 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92f911d1-972a-4024-9cd7-311529a59fb7-system-cni-dir\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.865114 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.864776 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/92f911d1-972a-4024-9cd7-311529a59fb7-os-release\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.865114 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.864860 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.865114 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.864901 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f344540f-2677-49dd-8a0f-106be00f414d-env-overrides\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.865114 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.865093 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 15 09:18:37.866253 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.865355 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lj2vn\"" Jun 15 09:18:37.866253 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.865803 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 15 09:18:37.866623 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.866605 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jlhbm" Jun 15 09:18:37.868692 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.868610 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-s7dtp\"" Jun 15 09:18:37.869128 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.869111 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:18:37.869227 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:37.869180 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdmf7" podUID="4e83c2a6-9354-4e58-85fc-9f351b1991d7" Jun 15 09:18:37.869516 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.869495 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 15 09:18:37.869602 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.869547 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 15 09:18:37.871741 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.871724 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:37.871859 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:37.871780 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kddf5" podUID="56a0c215-5d6d-4cb2-b9aa-505333bc0cea" Jun 15 09:18:37.882280 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.882223 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lpvlf" Jun 15 09:18:37.889704 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.889380 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lpvlf" Jun 15 09:18:37.934387 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.934347 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-143.ec2.internal" event={"ID":"6546904cab304243cc28b9f8f6f2abcd","Type":"ContainerStarted","Data":"e2975b1d0eab747d1e956fffcc282445d18ef23e0eea4418f7ce1f81d27d8430"} Jun 15 09:18:37.935512 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.935493 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-143.ec2.internal" event={"ID":"5f6ea51b0527deb3dc5845d11ce6220c","Type":"ContainerStarted","Data":"ac08d224661ae06a6833acb9cada4d104739e7378a7c8185ecafca56a15d61a9"} Jun 15 09:18:37.953253 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.953238 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 15 09:18:37.965808 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.965788 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-etc-systemd\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:37.965920 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.965816 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-var-lib-kubelet\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:37.965920 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.965862 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-system-cni-dir\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:37.965920 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.965881 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-run-openvswitch\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.965920 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.965903 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f344540f-2677-49dd-8a0f-106be00f414d-ovn-node-metrics-cert\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.966119 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.965930 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/facd9737-c7d9-4fa5-9ece-f598f2c0cf1f-device-dir\") pod \"aws-ebs-csi-driver-node-k58cq\" (UID: \"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" Jun 15 09:18:37.966119 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.965994 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/facd9737-c7d9-4fa5-9ece-f598f2c0cf1f-device-dir\") pod \"aws-ebs-csi-driver-node-k58cq\" (UID: \"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" Jun 15 09:18:37.966119 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.965994 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-run-openvswitch\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.966223 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966120 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/facd9737-c7d9-4fa5-9ece-f598f2c0cf1f-sys-fs\") pod \"aws-ebs-csi-driver-node-k58cq\" (UID: \"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" Jun 15 09:18:37.966223 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966145 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2699a480-5720-4d12-ab42-7c30a489e7d7-multus-daemon-config\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:37.966223 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966180 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7a4b513-5f5a-49ca-b112-6c4c00223f04-host-slash\") pod \"iptables-alerter-p6nnr\" (UID: \"d7a4b513-5f5a-49ca-b112-6c4c00223f04\") " pod="openshift-network-operator/iptables-alerter-p6nnr" Jun 15 09:18:37.966223 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966201 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-run-systemd\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.966223 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966201 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/facd9737-c7d9-4fa5-9ece-f598f2c0cf1f-sys-fs\") pod \"aws-ebs-csi-driver-node-k58cq\" (UID: \"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" Jun 15 09:18:37.966223 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966215 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-host-cni-netd\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.966469 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966248 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7a4b513-5f5a-49ca-b112-6c4c00223f04-host-slash\") pod \"iptables-alerter-p6nnr\" (UID: \"d7a4b513-5f5a-49ca-b112-6c4c00223f04\") " pod="openshift-network-operator/iptables-alerter-p6nnr" Jun 15 09:18:37.966469 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966278 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-etc-kubernetes\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:37.966469 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966274 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-run-systemd\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.966469 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966279 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-host-cni-netd\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.966469 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966308 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-multus-cni-dir\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:37.966469 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966340 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfdp5\" (UniqueName: \"kubernetes.io/projected/eac64cde-28f4-49be-8543-42e3b4ed898f-kube-api-access-cfdp5\") pod \"node-resolver-jlhbm\" (UID: \"eac64cde-28f4-49be-8543-42e3b4ed898f\") " pod="openshift-dns/node-resolver-jlhbm" Jun 15 09:18:37.966469 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966367 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-log-socket\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.966469 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966393 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f344540f-2677-49dd-8a0f-106be00f414d-ovnkube-script-lib\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.966469 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966417 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-etc-sysconfig\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:37.966469 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966441 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-host\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:37.966469 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966466 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-host-run-netns\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:37.967025 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966442 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-log-socket\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.967025 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966552 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-dbus\") pod \"global-pull-secret-syncer-kddf5\" (UID: \"56a0c215-5d6d-4cb2-b9aa-505333bc0cea\") " pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:37.967025 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966598 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsbvm\" (UniqueName: \"kubernetes.io/projected/a11b7428-db5b-4672-97f0-4cfaf32de873-kube-api-access-nsbvm\") pod \"node-ca-lfzxm\" (UID: \"a11b7428-db5b-4672-97f0-4cfaf32de873\") " pod="openshift-image-registry/node-ca-lfzxm" Jun 15 09:18:37.967025 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966607 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 15 09:18:37.967025 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966666 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eac64cde-28f4-49be-8543-42e3b4ed898f-hosts-file\") pod \"node-resolver-jlhbm\" (UID: \"eac64cde-28f4-49be-8543-42e3b4ed898f\") " pod="openshift-dns/node-resolver-jlhbm" Jun 15 09:18:37.967025 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966701 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-run-ovn\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.967025 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966725 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/92f911d1-972a-4024-9cd7-311529a59fb7-os-release\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.967025 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966744 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-run\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:37.967025 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.966771 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2sjk\" (UniqueName: \"kubernetes.io/projected/449bde46-c22c-4d96-9b35-52c9130e47df-kube-api-access-c2sjk\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:37.967025 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.967012 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f344540f-2677-49dd-8a0f-106be00f414d-env-overrides\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.967492 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.967044 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-host-kubelet\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.967492 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.967069 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92f911d1-972a-4024-9cd7-311529a59fb7-cnibin\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.967492 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.967070 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/92f911d1-972a-4024-9cd7-311529a59fb7-os-release\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.967492 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.967090 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-lib-modules\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:37.967492 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.967136 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-run-ovn\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.967492 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.967152 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2699a480-5720-4d12-ab42-7c30a489e7d7-cni-binary-copy\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:37.967492 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.967194 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dd2e6f0c-a1ba-4f06-8286-ff69fdf52440-agent-certs\") pod \"konnectivity-agent-47q6n\" (UID: \"dd2e6f0c-a1ba-4f06-8286-ff69fdf52440\") " pod="kube-system/konnectivity-agent-47q6n" Jun 15 09:18:37.967492 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.967225 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d7a4b513-5f5a-49ca-b112-6c4c00223f04-iptables-alerter-script\") pod \"iptables-alerter-p6nnr\" (UID: \"d7a4b513-5f5a-49ca-b112-6c4c00223f04\") " pod="openshift-network-operator/iptables-alerter-p6nnr" Jun 15 09:18:37.967492 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.967249 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-etc-modprobe-d\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:37.967492 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.967275 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-etc-sysctl-d\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:37.967492 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.967327 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-host-var-lib-cni-multus\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:37.967492 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.967353 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqnwc\" (UniqueName: \"kubernetes.io/projected/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-kube-api-access-vqnwc\") pod \"network-metrics-daemon-7trxt\" (UID: \"67c71a8c-71f7-4494-a8cd-fd7b2af8ab51\") " pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:18:37.967492 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.967365 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f344540f-2677-49dd-8a0f-106be00f414d-ovnkube-script-lib\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.967492 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.967379 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g8h6\" (UniqueName: \"kubernetes.io/projected/d7a4b513-5f5a-49ca-b112-6c4c00223f04-kube-api-access-2g8h6\") pod \"iptables-alerter-p6nnr\" (UID: \"d7a4b513-5f5a-49ca-b112-6c4c00223f04\") " pod="openshift-network-operator/iptables-alerter-p6nnr" Jun 15 09:18:37.968111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.967471 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92f911d1-972a-4024-9cd7-311529a59fb7-cnibin\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.968111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.967611 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-host-kubelet\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.968111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.967644 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6dzb\" (UniqueName: \"kubernetes.io/projected/f344540f-2677-49dd-8a0f-106be00f414d-kube-api-access-f6dzb\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.968111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.967865 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-multus-conf-dir\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:37.968111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.967967 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a11b7428-db5b-4672-97f0-4cfaf32de873-serviceca\") pod \"node-ca-lfzxm\" (UID: \"a11b7428-db5b-4672-97f0-4cfaf32de873\") " pod="openshift-image-registry/node-ca-lfzxm" Jun 15 09:18:37.968111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.968028 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f344540f-2677-49dd-8a0f-106be00f414d-env-overrides\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.968111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.968092 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eac64cde-28f4-49be-8543-42e3b4ed898f-tmp-dir\") pod \"node-resolver-jlhbm\" (UID: \"eac64cde-28f4-49be-8543-42e3b4ed898f\") " pod="openshift-dns/node-resolver-jlhbm" Jun 15 09:18:37.968429 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.968165 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-etc-openvswitch\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.968429 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.968210 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92f911d1-972a-4024-9cd7-311529a59fb7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.968429 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.968220 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-etc-openvswitch\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.968429 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.968251 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/facd9737-c7d9-4fa5-9ece-f598f2c0cf1f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k58cq\" (UID: \"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" Jun 15 09:18:37.968429 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.968366 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d7a4b513-5f5a-49ca-b112-6c4c00223f04-iptables-alerter-script\") pod \"iptables-alerter-p6nnr\" (UID: \"d7a4b513-5f5a-49ca-b112-6c4c00223f04\") " pod="openshift-network-operator/iptables-alerter-p6nnr" Jun 15 09:18:37.968429 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.968395 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/facd9737-c7d9-4fa5-9ece-f598f2c0cf1f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k58cq\" (UID: \"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" Jun 15 09:18:37.968429 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.968395 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92f911d1-972a-4024-9cd7-311529a59fb7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.968752 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.968425 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-sys\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:37.968752 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.968493 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-os-release\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:37.968752 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.968528 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-multus-socket-dir-parent\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:37.968752 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.968550 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2vs8\" (UniqueName: \"kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8\") pod \"network-check-target-cdmf7\" (UID: \"4e83c2a6-9354-4e58-85fc-9f351b1991d7\") " pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:18:37.968752 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.968576 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs\") pod \"network-metrics-daemon-7trxt\" (UID: \"67c71a8c-71f7-4494-a8cd-fd7b2af8ab51\") " pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:18:37.968752 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.968608 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/facd9737-c7d9-4fa5-9ece-f598f2c0cf1f-etc-selinux\") pod \"aws-ebs-csi-driver-node-k58cq\" (UID: \"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" Jun 15 09:18:37.968752 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.968635 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/449bde46-c22c-4d96-9b35-52c9130e47df-etc-tuned\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:37.968752 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.968693 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-host-run-k8s-cni-cncf-io\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:37.968752 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:37.968743 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 09:18:37.968752 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.968747 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/facd9737-c7d9-4fa5-9ece-f598f2c0cf1f-etc-selinux\") pod \"aws-ebs-csi-driver-node-k58cq\" (UID: \"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" Jun 15 09:18:37.969221 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.968756 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-systemd-units\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.969221 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.968796 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92f911d1-972a-4024-9cd7-311529a59fb7-cni-binary-copy\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.969221 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.968806 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-systemd-units\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.969221 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:37.968950 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs podName:67c71a8c-71f7-4494-a8cd-fd7b2af8ab51 nodeName:}" failed. No retries permitted until 2026-06-15 09:18:38.468900214 +0000 UTC m=+3.032322508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs") pod "network-metrics-daemon-7trxt" (UID: "67c71a8c-71f7-4494-a8cd-fd7b2af8ab51") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 09:18:37.969221 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.968971 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-cnibin\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:37.969221 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969000 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-host-slash\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.969221 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969025 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92f911d1-972a-4024-9cd7-311529a59fb7-system-cni-dir\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.969221 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969062 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/449bde46-c22c-4d96-9b35-52c9130e47df-tmp\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:37.969221 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969087 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-host-slash\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.969221 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969099 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-host-var-lib-cni-bin\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:37.969221 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969141 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-host-var-lib-kubelet\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:37.969221 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969147 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92f911d1-972a-4024-9cd7-311529a59fb7-system-cni-dir\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.969221 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969167 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-hostroot\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:37.969861 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969230 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-host-run-multus-certs\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:37.969861 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969273 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.969861 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969323 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w8ks\" (UniqueName: \"kubernetes.io/projected/92f911d1-972a-4024-9cd7-311529a59fb7-kube-api-access-9w8ks\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.969861 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969345 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.969861 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969364 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92f911d1-972a-4024-9cd7-311529a59fb7-cni-binary-copy\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.969861 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969436 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-etc-kubernetes\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:37.969861 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969494 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvpxv\" (UniqueName: \"kubernetes.io/projected/2699a480-5720-4d12-ab42-7c30a489e7d7-kube-api-access-xvpxv\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:37.969861 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969536 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-kubelet-config\") pod \"global-pull-secret-syncer-kddf5\" (UID: \"56a0c215-5d6d-4cb2-b9aa-505333bc0cea\") " pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:37.969861 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969606 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-node-log\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.969861 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969633 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-host-cni-bin\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.969861 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969669 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-node-log\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.969861 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969721 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/92f911d1-972a-4024-9cd7-311529a59fb7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.969861 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969762 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-host-cni-bin\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.969861 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969820 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-etc-sysctl-conf\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:37.970420 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969867 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dd2e6f0c-a1ba-4f06-8286-ff69fdf52440-konnectivity-ca\") pod \"konnectivity-agent-47q6n\" (UID: \"dd2e6f0c-a1ba-4f06-8286-ff69fdf52440\") " pod="kube-system/konnectivity-agent-47q6n" Jun 15 09:18:37.970420 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969888 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a11b7428-db5b-4672-97f0-4cfaf32de873-host\") pod \"node-ca-lfzxm\" (UID: \"a11b7428-db5b-4672-97f0-4cfaf32de873\") " pod="openshift-image-registry/node-ca-lfzxm" Jun 15 09:18:37.970420 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969921 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-var-lib-openvswitch\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.970420 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969967 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-host-run-ovn-kubernetes\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.970420 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.969964 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-var-lib-openvswitch\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.970420 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.970018 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f344540f-2677-49dd-8a0f-106be00f414d-ovnkube-config\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.970420 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.970041 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z4tq\" (UniqueName: \"kubernetes.io/projected/facd9737-c7d9-4fa5-9ece-f598f2c0cf1f-kube-api-access-6z4tq\") pod \"aws-ebs-csi-driver-node-k58cq\" (UID: \"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" Jun 15 09:18:37.970420 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.970064 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret\") pod \"global-pull-secret-syncer-kddf5\" (UID: \"56a0c215-5d6d-4cb2-b9aa-505333bc0cea\") " pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:37.970420 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.970071 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-host-run-ovn-kubernetes\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.970420 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.970088 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-host-run-netns\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.970420 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.970164 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/92f911d1-972a-4024-9cd7-311529a59fb7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.970420 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.970208 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/facd9737-c7d9-4fa5-9ece-f598f2c0cf1f-socket-dir\") pod \"aws-ebs-csi-driver-node-k58cq\" (UID: \"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" Jun 15 09:18:37.970420 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.970232 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/facd9737-c7d9-4fa5-9ece-f598f2c0cf1f-registration-dir\") pod \"aws-ebs-csi-driver-node-k58cq\" (UID: \"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" Jun 15 09:18:37.970420 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.970281 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/92f911d1-972a-4024-9cd7-311529a59fb7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.971003 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.970285 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f344540f-2677-49dd-8a0f-106be00f414d-host-run-netns\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.971003 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.970401 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/facd9737-c7d9-4fa5-9ece-f598f2c0cf1f-registration-dir\") pod \"aws-ebs-csi-driver-node-k58cq\" (UID: \"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" Jun 15 09:18:37.971003 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.970532 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/facd9737-c7d9-4fa5-9ece-f598f2c0cf1f-socket-dir\") pod \"aws-ebs-csi-driver-node-k58cq\" (UID: \"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" Jun 15 09:18:37.971003 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.970654 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f344540f-2677-49dd-8a0f-106be00f414d-ovnkube-config\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.971003 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.970933 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/92f911d1-972a-4024-9cd7-311529a59fb7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:37.971325 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.971300 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f344540f-2677-49dd-8a0f-106be00f414d-ovn-node-metrics-cert\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.976427 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.976289 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqnwc\" (UniqueName: \"kubernetes.io/projected/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-kube-api-access-vqnwc\") pod \"network-metrics-daemon-7trxt\" (UID: \"67c71a8c-71f7-4494-a8cd-fd7b2af8ab51\") " pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:18:37.977096 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.977061 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6dzb\" (UniqueName: \"kubernetes.io/projected/f344540f-2677-49dd-8a0f-106be00f414d-kube-api-access-f6dzb\") pod \"ovnkube-node-pxdxj\" (UID: \"f344540f-2677-49dd-8a0f-106be00f414d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:37.977404 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.977378 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g8h6\" (UniqueName: \"kubernetes.io/projected/d7a4b513-5f5a-49ca-b112-6c4c00223f04-kube-api-access-2g8h6\") pod \"iptables-alerter-p6nnr\" (UID: \"d7a4b513-5f5a-49ca-b112-6c4c00223f04\") " pod="openshift-network-operator/iptables-alerter-p6nnr" Jun 15 09:18:37.978545 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.978524 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z4tq\" (UniqueName: \"kubernetes.io/projected/facd9737-c7d9-4fa5-9ece-f598f2c0cf1f-kube-api-access-6z4tq\") pod \"aws-ebs-csi-driver-node-k58cq\" (UID: \"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" Jun 15 09:18:37.978908 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:37.978892 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w8ks\" (UniqueName: \"kubernetes.io/projected/92f911d1-972a-4024-9cd7-311529a59fb7-kube-api-access-9w8ks\") pod \"multus-additional-cni-plugins-44gsl\" (UID: \"92f911d1-972a-4024-9cd7-311529a59fb7\") " pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:38.070659 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.070634 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-lib-modules\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.070785 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.070668 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2699a480-5720-4d12-ab42-7c30a489e7d7-cni-binary-copy\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.070785 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.070692 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dd2e6f0c-a1ba-4f06-8286-ff69fdf52440-agent-certs\") pod \"konnectivity-agent-47q6n\" (UID: \"dd2e6f0c-a1ba-4f06-8286-ff69fdf52440\") " pod="kube-system/konnectivity-agent-47q6n" Jun 15 09:18:38.070785 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.070713 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-etc-modprobe-d\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.070785 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.070734 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-etc-sysctl-d\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.070785 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.070759 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-host-var-lib-cni-multus\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.070785 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.070779 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-multus-conf-dir\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.071111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.070793 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a11b7428-db5b-4672-97f0-4cfaf32de873-serviceca\") pod \"node-ca-lfzxm\" (UID: \"a11b7428-db5b-4672-97f0-4cfaf32de873\") " pod="openshift-image-registry/node-ca-lfzxm" Jun 15 09:18:38.071111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.070809 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eac64cde-28f4-49be-8543-42e3b4ed898f-tmp-dir\") pod \"node-resolver-jlhbm\" (UID: \"eac64cde-28f4-49be-8543-42e3b4ed898f\") " pod="openshift-dns/node-resolver-jlhbm" Jun 15 09:18:38.071111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.070824 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-sys\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.071111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.070828 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-lib-modules\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.071111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.070856 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-os-release\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.071111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.070906 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-multus-socket-dir-parent\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.071111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.070907 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-multus-conf-dir\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.071111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.070923 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2vs8\" (UniqueName: \"kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8\") pod \"network-check-target-cdmf7\" (UID: \"4e83c2a6-9354-4e58-85fc-9f351b1991d7\") " pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:18:38.071111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.070953 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/449bde46-c22c-4d96-9b35-52c9130e47df-etc-tuned\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.071111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.070967 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-host-run-k8s-cni-cncf-io\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.071111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.070981 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-cnibin\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.071111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.070990 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-etc-modprobe-d\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.071111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.070995 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/449bde46-c22c-4d96-9b35-52c9130e47df-tmp\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.071111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071032 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-host-var-lib-cni-bin\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.071111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071060 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-host-var-lib-kubelet\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.071852 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071083 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-hostroot\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.071852 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071106 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-host-run-multus-certs\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.071852 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071148 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-etc-kubernetes\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.071852 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071166 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-host-var-lib-kubelet\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.071852 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071182 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-host-var-lib-cni-bin\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.071852 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071185 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvpxv\" (UniqueName: \"kubernetes.io/projected/2699a480-5720-4d12-ab42-7c30a489e7d7-kube-api-access-xvpxv\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.071852 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071218 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-kubelet-config\") pod \"global-pull-secret-syncer-kddf5\" (UID: \"56a0c215-5d6d-4cb2-b9aa-505333bc0cea\") " pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:38.071852 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071227 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-etc-kubernetes\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.071852 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071219 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-host-run-multus-certs\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.071852 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071155 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-hostroot\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.071852 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071173 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-multus-socket-dir-parent\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.071852 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071264 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2699a480-5720-4d12-ab42-7c30a489e7d7-cni-binary-copy\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.071852 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071282 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-kubelet-config\") pod \"global-pull-secret-syncer-kddf5\" (UID: \"56a0c215-5d6d-4cb2-b9aa-505333bc0cea\") " pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:38.071852 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071297 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-sys\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.071852 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071352 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-cnibin\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.072504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071380 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-os-release\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.072504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071379 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-etc-sysctl-conf\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.072504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071408 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-etc-sysctl-d\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.072504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071422 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dd2e6f0c-a1ba-4f06-8286-ff69fdf52440-konnectivity-ca\") pod \"konnectivity-agent-47q6n\" (UID: \"dd2e6f0c-a1ba-4f06-8286-ff69fdf52440\") " pod="kube-system/konnectivity-agent-47q6n" Jun 15 09:18:38.072504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071448 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a11b7428-db5b-4672-97f0-4cfaf32de873-host\") pod \"node-ca-lfzxm\" (UID: \"a11b7428-db5b-4672-97f0-4cfaf32de873\") " pod="openshift-image-registry/node-ca-lfzxm" Jun 15 09:18:38.072504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071475 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret\") pod \"global-pull-secret-syncer-kddf5\" (UID: \"56a0c215-5d6d-4cb2-b9aa-505333bc0cea\") " pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:38.072504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071484 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-etc-sysctl-conf\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.072504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071503 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-etc-systemd\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.072504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071524 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-host-run-k8s-cni-cncf-io\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.072504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071525 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eac64cde-28f4-49be-8543-42e3b4ed898f-tmp-dir\") pod \"node-resolver-jlhbm\" (UID: \"eac64cde-28f4-49be-8543-42e3b4ed898f\") " pod="openshift-dns/node-resolver-jlhbm" Jun 15 09:18:38.072504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071526 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-var-lib-kubelet\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.072504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071567 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-system-cni-dir\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.072504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071573 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-var-lib-kubelet\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.072504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071585 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a11b7428-db5b-4672-97f0-4cfaf32de873-serviceca\") pod \"node-ca-lfzxm\" (UID: \"a11b7428-db5b-4672-97f0-4cfaf32de873\") " pod="openshift-image-registry/node-ca-lfzxm" Jun 15 09:18:38.072504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071616 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a11b7428-db5b-4672-97f0-4cfaf32de873-host\") pod \"node-ca-lfzxm\" (UID: \"a11b7428-db5b-4672-97f0-4cfaf32de873\") " pod="openshift-image-registry/node-ca-lfzxm" Jun 15 09:18:38.073031 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071619 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2699a480-5720-4d12-ab42-7c30a489e7d7-multus-daemon-config\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.073031 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071628 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-etc-systemd\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.073031 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071658 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-system-cni-dir\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.073031 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071675 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-host-var-lib-cni-multus\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.073031 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:38.071695 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 09:18:38.073031 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071705 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-etc-kubernetes\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.073031 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:38.071767 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret podName:56a0c215-5d6d-4cb2-b9aa-505333bc0cea nodeName:}" failed. No retries permitted until 2026-06-15 09:18:38.571747537 +0000 UTC m=+3.135169834 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret") pod "global-pull-secret-syncer-kddf5" (UID: "56a0c215-5d6d-4cb2-b9aa-505333bc0cea") : object "kube-system"/"original-pull-secret" not registered Jun 15 09:18:38.073031 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071805 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-multus-cni-dir\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.073031 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071808 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-etc-kubernetes\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.073031 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071845 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfdp5\" (UniqueName: \"kubernetes.io/projected/eac64cde-28f4-49be-8543-42e3b4ed898f-kube-api-access-cfdp5\") pod \"node-resolver-jlhbm\" (UID: \"eac64cde-28f4-49be-8543-42e3b4ed898f\") " pod="openshift-dns/node-resolver-jlhbm" Jun 15 09:18:38.073031 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071871 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-etc-sysconfig\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.073031 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071892 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-host\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.073031 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071920 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-host-run-netns\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.073031 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071943 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-dbus\") pod \"global-pull-secret-syncer-kddf5\" (UID: \"56a0c215-5d6d-4cb2-b9aa-505333bc0cea\") " pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:38.073031 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071945 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-multus-cni-dir\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.073582 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071986 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-etc-sysconfig\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.073582 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071998 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-dbus\") pod \"global-pull-secret-syncer-kddf5\" (UID: \"56a0c215-5d6d-4cb2-b9aa-505333bc0cea\") " pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:38.073582 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.071993 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-host\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.073582 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.072031 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsbvm\" (UniqueName: \"kubernetes.io/projected/a11b7428-db5b-4672-97f0-4cfaf32de873-kube-api-access-nsbvm\") pod \"node-ca-lfzxm\" (UID: \"a11b7428-db5b-4672-97f0-4cfaf32de873\") " pod="openshift-image-registry/node-ca-lfzxm" Jun 15 09:18:38.073582 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.072048 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2699a480-5720-4d12-ab42-7c30a489e7d7-host-run-netns\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.073582 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.072058 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eac64cde-28f4-49be-8543-42e3b4ed898f-hosts-file\") pod \"node-resolver-jlhbm\" (UID: \"eac64cde-28f4-49be-8543-42e3b4ed898f\") " pod="openshift-dns/node-resolver-jlhbm" Jun 15 09:18:38.073582 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.072085 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-run\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.073582 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.072108 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2sjk\" (UniqueName: \"kubernetes.io/projected/449bde46-c22c-4d96-9b35-52c9130e47df-kube-api-access-c2sjk\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.073582 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.072123 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2699a480-5720-4d12-ab42-7c30a489e7d7-multus-daemon-config\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.073582 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.072162 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dd2e6f0c-a1ba-4f06-8286-ff69fdf52440-konnectivity-ca\") pod \"konnectivity-agent-47q6n\" (UID: \"dd2e6f0c-a1ba-4f06-8286-ff69fdf52440\") " pod="kube-system/konnectivity-agent-47q6n" Jun 15 09:18:38.073582 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.072179 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eac64cde-28f4-49be-8543-42e3b4ed898f-hosts-file\") pod \"node-resolver-jlhbm\" (UID: \"eac64cde-28f4-49be-8543-42e3b4ed898f\") " pod="openshift-dns/node-resolver-jlhbm" Jun 15 09:18:38.073582 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.072228 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/449bde46-c22c-4d96-9b35-52c9130e47df-run\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.074033 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.073774 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/449bde46-c22c-4d96-9b35-52c9130e47df-tmp\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.074266 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.074241 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/449bde46-c22c-4d96-9b35-52c9130e47df-etc-tuned\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.074390 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.074369 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dd2e6f0c-a1ba-4f06-8286-ff69fdf52440-agent-certs\") pod \"konnectivity-agent-47q6n\" (UID: \"dd2e6f0c-a1ba-4f06-8286-ff69fdf52440\") " pod="kube-system/konnectivity-agent-47q6n" Jun 15 09:18:38.077806 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:38.077786 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 09:18:38.077914 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:38.077811 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 09:18:38.077914 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:38.077827 2562 projected.go:196] Error preparing data for projected volume kube-api-access-x2vs8 for pod openshift-network-diagnostics/network-check-target-cdmf7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 09:18:38.077914 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:38.077900 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8 podName:4e83c2a6-9354-4e58-85fc-9f351b1991d7 nodeName:}" failed. No retries permitted until 2026-06-15 09:18:38.577887939 +0000 UTC m=+3.141310241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-x2vs8" (UniqueName: "kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8") pod "network-check-target-cdmf7" (UID: "4e83c2a6-9354-4e58-85fc-9f351b1991d7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 09:18:38.079793 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.079776 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvpxv\" (UniqueName: \"kubernetes.io/projected/2699a480-5720-4d12-ab42-7c30a489e7d7-kube-api-access-xvpxv\") pod \"multus-sxdtv\" (UID: \"2699a480-5720-4d12-ab42-7c30a489e7d7\") " pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.080350 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.080314 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsbvm\" (UniqueName: \"kubernetes.io/projected/a11b7428-db5b-4672-97f0-4cfaf32de873-kube-api-access-nsbvm\") pod \"node-ca-lfzxm\" (UID: \"a11b7428-db5b-4672-97f0-4cfaf32de873\") " pod="openshift-image-registry/node-ca-lfzxm" Jun 15 09:18:38.080629 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.080613 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2sjk\" (UniqueName: \"kubernetes.io/projected/449bde46-c22c-4d96-9b35-52c9130e47df-kube-api-access-c2sjk\") pod \"tuned-bkgdl\" (UID: \"449bde46-c22c-4d96-9b35-52c9130e47df\") " pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.080722 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.080701 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfdp5\" (UniqueName: \"kubernetes.io/projected/eac64cde-28f4-49be-8543-42e3b4ed898f-kube-api-access-cfdp5\") pod \"node-resolver-jlhbm\" (UID: \"eac64cde-28f4-49be-8543-42e3b4ed898f\") " pod="openshift-dns/node-resolver-jlhbm" Jun 15 09:18:38.155018 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.154962 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-44gsl" Jun 15 09:18:38.165391 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.165375 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" Jun 15 09:18:38.177135 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.177114 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p6nnr" Jun 15 09:18:38.182796 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.182780 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:18:38.190450 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.190433 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-47q6n" Jun 15 09:18:38.194480 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.194464 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lfzxm" Jun 15 09:18:38.202038 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.202022 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" Jun 15 09:18:38.209430 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.209412 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sxdtv" Jun 15 09:18:38.214075 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.214060 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jlhbm" Jun 15 09:18:38.474607 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.474547 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs\") pod \"network-metrics-daemon-7trxt\" (UID: \"67c71a8c-71f7-4494-a8cd-fd7b2af8ab51\") " pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:18:38.474759 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:38.474672 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 09:18:38.474759 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:38.474739 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs podName:67c71a8c-71f7-4494-a8cd-fd7b2af8ab51 nodeName:}" failed. No retries permitted until 2026-06-15 09:18:39.47471913 +0000 UTC m=+4.038141415 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs") pod "network-metrics-daemon-7trxt" (UID: "67c71a8c-71f7-4494-a8cd-fd7b2af8ab51") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 09:18:38.575192 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.575162 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret\") pod \"global-pull-secret-syncer-kddf5\" (UID: \"56a0c215-5d6d-4cb2-b9aa-505333bc0cea\") " pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:38.575332 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:38.575287 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 09:18:38.575405 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:38.575346 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret podName:56a0c215-5d6d-4cb2-b9aa-505333bc0cea nodeName:}" failed. No retries permitted until 2026-06-15 09:18:39.575331902 +0000 UTC m=+4.138754184 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret") pod "global-pull-secret-syncer-kddf5" (UID: "56a0c215-5d6d-4cb2-b9aa-505333bc0cea") : object "kube-system"/"original-pull-secret" not registered Jun 15 09:18:38.676446 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.676412 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2vs8\" (UniqueName: \"kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8\") pod \"network-check-target-cdmf7\" (UID: \"4e83c2a6-9354-4e58-85fc-9f351b1991d7\") " pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:18:38.676594 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:38.676555 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 09:18:38.676594 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:38.676575 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 09:18:38.676594 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:38.676586 2562 projected.go:196] Error preparing data for projected volume kube-api-access-x2vs8 for pod openshift-network-diagnostics/network-check-target-cdmf7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 09:18:38.676751 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:38.676645 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8 podName:4e83c2a6-9354-4e58-85fc-9f351b1991d7 nodeName:}" failed. No retries permitted until 2026-06-15 09:18:39.676625847 +0000 UTC m=+4.240048131 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-x2vs8" (UniqueName: "kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8") pod "network-check-target-cdmf7" (UID: "4e83c2a6-9354-4e58-85fc-9f351b1991d7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 09:18:38.880750 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:38.880713 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7a4b513_5f5a_49ca_b112_6c4c00223f04.slice/crio-4fb3e025a81fe2b800edea175c378df1b271d353e074dc54b1e3bad5b128fad3 WatchSource:0}: Error finding container 4fb3e025a81fe2b800edea175c378df1b271d353e074dc54b1e3bad5b128fad3: Status 404 returned error can't find the container with id 4fb3e025a81fe2b800edea175c378df1b271d353e074dc54b1e3bad5b128fad3 Jun 15 09:18:38.882258 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:38.882231 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfacd9737_c7d9_4fa5_9ece_f598f2c0cf1f.slice/crio-8b6c3b392dc875be05973c82255bf13bd0a3403560527ab0186e0aaf8a86b371 WatchSource:0}: Error finding container 8b6c3b392dc875be05973c82255bf13bd0a3403560527ab0186e0aaf8a86b371: Status 404 returned error can't find the container with id 8b6c3b392dc875be05973c82255bf13bd0a3403560527ab0186e0aaf8a86b371 Jun 15 09:18:38.887113 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:38.886356 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92f911d1_972a_4024_9cd7_311529a59fb7.slice/crio-42ddbb750322c3e0d4ca9a577bafb543234db1af12b5d184af63fdeb11291d28 WatchSource:0}: Error finding container 42ddbb750322c3e0d4ca9a577bafb543234db1af12b5d184af63fdeb11291d28: Status 404 returned error can't find the container with id 42ddbb750322c3e0d4ca9a577bafb543234db1af12b5d184af63fdeb11291d28 Jun 15 09:18:38.888644 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:38.888625 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2699a480_5720_4d12_ab42_7c30a489e7d7.slice/crio-3d6630b703ee81f4ce75739d63c9f5a07d739e22783fd144a0096378d6dee27a WatchSource:0}: Error finding container 3d6630b703ee81f4ce75739d63c9f5a07d739e22783fd144a0096378d6dee27a: Status 404 returned error can't find the container with id 3d6630b703ee81f4ce75739d63c9f5a07d739e22783fd144a0096378d6dee27a Jun 15 09:18:38.890537 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:38.890085 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda11b7428_db5b_4672_97f0_4cfaf32de873.slice/crio-cca0b068561eb59e3cee8eee37e3edf14d8ba2e47113b2191501b5fa28e45156 WatchSource:0}: Error finding container cca0b068561eb59e3cee8eee37e3edf14d8ba2e47113b2191501b5fa28e45156: Status 404 returned error can't find the container with id cca0b068561eb59e3cee8eee37e3edf14d8ba2e47113b2191501b5fa28e45156 Jun 15 09:18:38.891296 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:38.891171 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod449bde46_c22c_4d96_9b35_52c9130e47df.slice/crio-8ba1d75b8beeeab98ec91e5614e3e8bfb37c48007ba21de0ad4bb3914b87d781 WatchSource:0}: Error finding container 8ba1d75b8beeeab98ec91e5614e3e8bfb37c48007ba21de0ad4bb3914b87d781: Status 404 returned error can't find the container with id 8ba1d75b8beeeab98ec91e5614e3e8bfb37c48007ba21de0ad4bb3914b87d781 Jun 15 09:18:38.892643 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.891814 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-14 09:13:37 +0000 UTC" deadline="2027-11-21 01:44:17.436762299 +0000 UTC" Jun 15 09:18:38.892643 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.891858 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12568h25m38.544908949s" Jun 15 09:18:38.892643 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:18:38.892505 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf344540f_2677_49dd_8a0f_106be00f414d.slice/crio-eb7ecb35f7637e266c8d030e00c2be6f0ef5c0dcc6c9a4b8b3236c9ef11e7839 WatchSource:0}: Error finding container eb7ecb35f7637e266c8d030e00c2be6f0ef5c0dcc6c9a4b8b3236c9ef11e7839: Status 404 returned error can't find the container with id eb7ecb35f7637e266c8d030e00c2be6f0ef5c0dcc6c9a4b8b3236c9ef11e7839 Jun 15 09:18:38.929754 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.929735 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:18:38.929859 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:38.929825 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdmf7" podUID="4e83c2a6-9354-4e58-85fc-9f351b1991d7" Jun 15 09:18:38.940537 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.940407 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" event={"ID":"f344540f-2677-49dd-8a0f-106be00f414d","Type":"ContainerStarted","Data":"eb7ecb35f7637e266c8d030e00c2be6f0ef5c0dcc6c9a4b8b3236c9ef11e7839"} Jun 15 09:18:38.942294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.942268 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-47q6n" event={"ID":"dd2e6f0c-a1ba-4f06-8286-ff69fdf52440","Type":"ContainerStarted","Data":"52b414448a600ace11383088e81ee84d6b5ef1be4d33abc0db663dd98c0b47ce"} Jun 15 09:18:38.943323 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.943301 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" event={"ID":"449bde46-c22c-4d96-9b35-52c9130e47df","Type":"ContainerStarted","Data":"8ba1d75b8beeeab98ec91e5614e3e8bfb37c48007ba21de0ad4bb3914b87d781"} Jun 15 09:18:38.944291 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.944271 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sxdtv" event={"ID":"2699a480-5720-4d12-ab42-7c30a489e7d7","Type":"ContainerStarted","Data":"3d6630b703ee81f4ce75739d63c9f5a07d739e22783fd144a0096378d6dee27a"} Jun 15 09:18:38.945239 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.945216 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jlhbm" event={"ID":"eac64cde-28f4-49be-8543-42e3b4ed898f","Type":"ContainerStarted","Data":"94219eafe7660ec00a5e08eaef18e460fd40ed7806620796336910c18f6af91e"} Jun 15 09:18:38.946163 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.946145 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" event={"ID":"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f","Type":"ContainerStarted","Data":"8b6c3b392dc875be05973c82255bf13bd0a3403560527ab0186e0aaf8a86b371"} Jun 15 09:18:38.947203 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.947183 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44gsl" event={"ID":"92f911d1-972a-4024-9cd7-311529a59fb7","Type":"ContainerStarted","Data":"42ddbb750322c3e0d4ca9a577bafb543234db1af12b5d184af63fdeb11291d28"} Jun 15 09:18:38.948669 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.948644 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p6nnr" event={"ID":"d7a4b513-5f5a-49ca-b112-6c4c00223f04","Type":"ContainerStarted","Data":"4fb3e025a81fe2b800edea175c378df1b271d353e074dc54b1e3bad5b128fad3"} Jun 15 09:18:38.952107 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:38.952087 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lfzxm" event={"ID":"a11b7428-db5b-4672-97f0-4cfaf32de873","Type":"ContainerStarted","Data":"cca0b068561eb59e3cee8eee37e3edf14d8ba2e47113b2191501b5fa28e45156"} Jun 15 09:18:39.484690 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:39.483936 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs\") pod \"network-metrics-daemon-7trxt\" (UID: \"67c71a8c-71f7-4494-a8cd-fd7b2af8ab51\") " pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:18:39.484690 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:39.484250 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 09:18:39.484690 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:39.484319 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs podName:67c71a8c-71f7-4494-a8cd-fd7b2af8ab51 nodeName:}" failed. No retries permitted until 2026-06-15 09:18:41.48429792 +0000 UTC m=+6.047720220 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs") pod "network-metrics-daemon-7trxt" (UID: "67c71a8c-71f7-4494-a8cd-fd7b2af8ab51") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 09:18:39.585119 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:39.585084 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret\") pod \"global-pull-secret-syncer-kddf5\" (UID: \"56a0c215-5d6d-4cb2-b9aa-505333bc0cea\") " pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:39.585290 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:39.585246 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 09:18:39.585349 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:39.585309 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret podName:56a0c215-5d6d-4cb2-b9aa-505333bc0cea nodeName:}" failed. No retries permitted until 2026-06-15 09:18:41.585290468 +0000 UTC m=+6.148712767 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret") pod "global-pull-secret-syncer-kddf5" (UID: "56a0c215-5d6d-4cb2-b9aa-505333bc0cea") : object "kube-system"/"original-pull-secret" not registered Jun 15 09:18:39.685959 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:39.685921 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2vs8\" (UniqueName: \"kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8\") pod \"network-check-target-cdmf7\" (UID: \"4e83c2a6-9354-4e58-85fc-9f351b1991d7\") " pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:18:39.686169 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:39.686130 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 09:18:39.686169 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:39.686152 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 09:18:39.686169 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:39.686166 2562 projected.go:196] Error preparing data for projected volume kube-api-access-x2vs8 for pod openshift-network-diagnostics/network-check-target-cdmf7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 09:18:39.686317 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:39.686220 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8 podName:4e83c2a6-9354-4e58-85fc-9f351b1991d7 nodeName:}" failed. No retries permitted until 2026-06-15 09:18:41.686202248 +0000 UTC m=+6.249624532 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-x2vs8" (UniqueName: "kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8") pod "network-check-target-cdmf7" (UID: "4e83c2a6-9354-4e58-85fc-9f351b1991d7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 09:18:39.892433 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:39.892321 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-14 09:13:37 +0000 UTC" deadline="2028-01-31 15:49:36.314093089 +0000 UTC" Jun 15 09:18:39.892433 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:39.892376 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14286h30m56.421720989s" Jun 15 09:18:39.931230 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:39.930949 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:39.931230 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:39.931073 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kddf5" podUID="56a0c215-5d6d-4cb2-b9aa-505333bc0cea" Jun 15 09:18:39.932831 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:39.932807 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:18:39.933027 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:39.932951 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7trxt" podUID="67c71a8c-71f7-4494-a8cd-fd7b2af8ab51" Jun 15 09:18:39.970255 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:39.970222 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-143.ec2.internal" event={"ID":"6546904cab304243cc28b9f8f6f2abcd","Type":"ContainerStarted","Data":"1c2b3b4f24b59a4ee700a2cfe48c304cc45a4365f278a7954f44ae2f3f95914b"} Jun 15 09:18:39.980306 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:39.980212 2562 generic.go:356] "Generic (PLEG): container finished" podID="5f6ea51b0527deb3dc5845d11ce6220c" containerID="aafbc749a315cf539b0a46d8517962faf91ed9d8210bb01424606afbea8bfd2d" exitCode=0 Jun 15 09:18:39.980306 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:39.980251 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-143.ec2.internal" event={"ID":"5f6ea51b0527deb3dc5845d11ce6220c","Type":"ContainerDied","Data":"aafbc749a315cf539b0a46d8517962faf91ed9d8210bb01424606afbea8bfd2d"} Jun 15 09:18:40.000134 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:40.000081 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-143.ec2.internal" podStartSLOduration=3.000061234 podStartE2EDuration="3.000061234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 09:18:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:18:39.985666293 +0000 UTC m=+4.549088602" watchObservedRunningTime="2026-06-15 09:18:40.000061234 +0000 UTC m=+4.563483572" Jun 15 09:18:40.929707 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:40.929664 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:18:40.930175 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:40.929813 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdmf7" podUID="4e83c2a6-9354-4e58-85fc-9f351b1991d7" Jun 15 09:18:41.000869 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:41.000662 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-143.ec2.internal" event={"ID":"5f6ea51b0527deb3dc5845d11ce6220c","Type":"ContainerStarted","Data":"1f60aabc4cd3a224bca8e181778fe0522a81eaa5f009f440b1a8c60fe4a82dfc"} Jun 15 09:18:41.502889 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:41.502851 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs\") pod \"network-metrics-daemon-7trxt\" (UID: \"67c71a8c-71f7-4494-a8cd-fd7b2af8ab51\") " pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:18:41.503071 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:41.503054 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 09:18:41.503135 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:41.503118 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs podName:67c71a8c-71f7-4494-a8cd-fd7b2af8ab51 nodeName:}" failed. No retries permitted until 2026-06-15 09:18:45.5030985 +0000 UTC m=+10.066520803 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs") pod "network-metrics-daemon-7trxt" (UID: "67c71a8c-71f7-4494-a8cd-fd7b2af8ab51") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 09:18:41.604643 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:41.604062 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret\") pod \"global-pull-secret-syncer-kddf5\" (UID: \"56a0c215-5d6d-4cb2-b9aa-505333bc0cea\") " pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:41.604643 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:41.604228 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 09:18:41.604643 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:41.604294 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret podName:56a0c215-5d6d-4cb2-b9aa-505333bc0cea nodeName:}" failed. No retries permitted until 2026-06-15 09:18:45.604273797 +0000 UTC m=+10.167696107 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret") pod "global-pull-secret-syncer-kddf5" (UID: "56a0c215-5d6d-4cb2-b9aa-505333bc0cea") : object "kube-system"/"original-pull-secret" not registered Jun 15 09:18:41.704456 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:41.704424 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2vs8\" (UniqueName: \"kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8\") pod \"network-check-target-cdmf7\" (UID: \"4e83c2a6-9354-4e58-85fc-9f351b1991d7\") " pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:18:41.704664 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:41.704644 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 09:18:41.704720 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:41.704677 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 09:18:41.704720 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:41.704691 2562 projected.go:196] Error preparing data for projected volume kube-api-access-x2vs8 for pod openshift-network-diagnostics/network-check-target-cdmf7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 09:18:41.704848 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:41.704760 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8 podName:4e83c2a6-9354-4e58-85fc-9f351b1991d7 nodeName:}" failed. No retries permitted until 2026-06-15 09:18:45.704739259 +0000 UTC m=+10.268161544 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-x2vs8" (UniqueName: "kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8") pod "network-check-target-cdmf7" (UID: "4e83c2a6-9354-4e58-85fc-9f351b1991d7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 09:18:41.932155 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:41.931589 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:41.932155 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:41.931718 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kddf5" podUID="56a0c215-5d6d-4cb2-b9aa-505333bc0cea" Jun 15 09:18:41.932634 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:41.932169 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:18:41.932634 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:41.932294 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7trxt" podUID="67c71a8c-71f7-4494-a8cd-fd7b2af8ab51" Jun 15 09:18:42.929723 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:42.929688 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:18:42.929924 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:42.929827 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdmf7" podUID="4e83c2a6-9354-4e58-85fc-9f351b1991d7" Jun 15 09:18:43.929508 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:43.929460 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:43.929508 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:43.929501 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:18:43.930038 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:43.929600 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kddf5" podUID="56a0c215-5d6d-4cb2-b9aa-505333bc0cea" Jun 15 09:18:43.930038 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:43.929741 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7trxt" podUID="67c71a8c-71f7-4494-a8cd-fd7b2af8ab51" Jun 15 09:18:44.930137 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:44.930102 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:18:44.930600 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:44.930238 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdmf7" podUID="4e83c2a6-9354-4e58-85fc-9f351b1991d7" Jun 15 09:18:45.541059 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:45.540993 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs\") pod \"network-metrics-daemon-7trxt\" (UID: \"67c71a8c-71f7-4494-a8cd-fd7b2af8ab51\") " pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:18:45.541251 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:45.541139 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 09:18:45.541251 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:45.541201 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs podName:67c71a8c-71f7-4494-a8cd-fd7b2af8ab51 nodeName:}" failed. No retries permitted until 2026-06-15 09:18:53.541181923 +0000 UTC m=+18.104604206 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs") pod "network-metrics-daemon-7trxt" (UID: "67c71a8c-71f7-4494-a8cd-fd7b2af8ab51") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 09:18:45.642208 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:45.642131 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret\") pod \"global-pull-secret-syncer-kddf5\" (UID: \"56a0c215-5d6d-4cb2-b9aa-505333bc0cea\") " pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:45.642375 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:45.642299 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 09:18:45.642375 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:45.642367 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret podName:56a0c215-5d6d-4cb2-b9aa-505333bc0cea nodeName:}" failed. No retries permitted until 2026-06-15 09:18:53.642347311 +0000 UTC m=+18.205769597 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret") pod "global-pull-secret-syncer-kddf5" (UID: "56a0c215-5d6d-4cb2-b9aa-505333bc0cea") : object "kube-system"/"original-pull-secret" not registered Jun 15 09:18:45.743353 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:45.743326 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2vs8\" (UniqueName: \"kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8\") pod \"network-check-target-cdmf7\" (UID: \"4e83c2a6-9354-4e58-85fc-9f351b1991d7\") " pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:18:45.743558 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:45.743473 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 09:18:45.743558 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:45.743488 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 09:18:45.743558 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:45.743496 2562 projected.go:196] Error preparing data for projected volume kube-api-access-x2vs8 for pod openshift-network-diagnostics/network-check-target-cdmf7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 09:18:45.743558 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:45.743536 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8 podName:4e83c2a6-9354-4e58-85fc-9f351b1991d7 nodeName:}" failed. No retries permitted until 2026-06-15 09:18:53.743523139 +0000 UTC m=+18.306945422 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-x2vs8" (UniqueName: "kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8") pod "network-check-target-cdmf7" (UID: "4e83c2a6-9354-4e58-85fc-9f351b1991d7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 09:18:45.933138 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:45.933107 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:18:45.933138 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:45.933121 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:45.933616 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:45.933251 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7trxt" podUID="67c71a8c-71f7-4494-a8cd-fd7b2af8ab51" Jun 15 09:18:45.933616 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:45.933367 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kddf5" podUID="56a0c215-5d6d-4cb2-b9aa-505333bc0cea" Jun 15 09:18:46.929783 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:46.929700 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:18:46.929957 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:46.929830 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdmf7" podUID="4e83c2a6-9354-4e58-85fc-9f351b1991d7" Jun 15 09:18:47.932924 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:47.932885 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:47.933383 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:47.933009 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kddf5" podUID="56a0c215-5d6d-4cb2-b9aa-505333bc0cea" Jun 15 09:18:47.933383 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:47.933267 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:18:47.933498 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:47.933379 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7trxt" podUID="67c71a8c-71f7-4494-a8cd-fd7b2af8ab51" Jun 15 09:18:48.929505 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:48.929470 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:18:48.929687 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:48.929595 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdmf7" podUID="4e83c2a6-9354-4e58-85fc-9f351b1991d7" Jun 15 09:18:49.929560 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:49.929531 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:49.929975 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:49.929567 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:18:49.929975 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:49.929675 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kddf5" podUID="56a0c215-5d6d-4cb2-b9aa-505333bc0cea" Jun 15 09:18:49.929975 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:49.929735 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7trxt" podUID="67c71a8c-71f7-4494-a8cd-fd7b2af8ab51" Jun 15 09:18:50.929848 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:50.929802 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:18:50.930341 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:50.929939 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdmf7" podUID="4e83c2a6-9354-4e58-85fc-9f351b1991d7" Jun 15 09:18:51.930290 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:51.930258 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:51.930750 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:51.930269 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:18:51.930750 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:51.930384 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kddf5" podUID="56a0c215-5d6d-4cb2-b9aa-505333bc0cea" Jun 15 09:18:51.930750 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:51.930499 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7trxt" podUID="67c71a8c-71f7-4494-a8cd-fd7b2af8ab51" Jun 15 09:18:52.929792 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:52.929759 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:18:52.929988 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:52.929877 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdmf7" podUID="4e83c2a6-9354-4e58-85fc-9f351b1991d7" Jun 15 09:18:53.602555 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:53.602520 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs\") pod \"network-metrics-daemon-7trxt\" (UID: \"67c71a8c-71f7-4494-a8cd-fd7b2af8ab51\") " pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:18:53.603030 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:53.602662 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 09:18:53.603030 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:53.602740 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs podName:67c71a8c-71f7-4494-a8cd-fd7b2af8ab51 nodeName:}" failed. No retries permitted until 2026-06-15 09:19:09.602716824 +0000 UTC m=+34.166139125 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs") pod "network-metrics-daemon-7trxt" (UID: "67c71a8c-71f7-4494-a8cd-fd7b2af8ab51") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 09:18:53.703051 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:53.703019 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret\") pod \"global-pull-secret-syncer-kddf5\" (UID: \"56a0c215-5d6d-4cb2-b9aa-505333bc0cea\") " pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:53.703211 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:53.703162 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 09:18:53.703258 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:53.703233 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret podName:56a0c215-5d6d-4cb2-b9aa-505333bc0cea nodeName:}" failed. No retries permitted until 2026-06-15 09:19:09.703211181 +0000 UTC m=+34.266633467 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret") pod "global-pull-secret-syncer-kddf5" (UID: "56a0c215-5d6d-4cb2-b9aa-505333bc0cea") : object "kube-system"/"original-pull-secret" not registered Jun 15 09:18:53.804037 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:53.803997 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2vs8\" (UniqueName: \"kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8\") pod \"network-check-target-cdmf7\" (UID: \"4e83c2a6-9354-4e58-85fc-9f351b1991d7\") " pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:18:53.804220 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:53.804172 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 09:18:53.804220 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:53.804199 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 09:18:53.804220 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:53.804212 2562 projected.go:196] Error preparing data for projected volume kube-api-access-x2vs8 for pod openshift-network-diagnostics/network-check-target-cdmf7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 09:18:53.804330 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:53.804275 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8 podName:4e83c2a6-9354-4e58-85fc-9f351b1991d7 nodeName:}" failed. No retries permitted until 2026-06-15 09:19:09.80425561 +0000 UTC m=+34.367677910 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-x2vs8" (UniqueName: "kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8") pod "network-check-target-cdmf7" (UID: "4e83c2a6-9354-4e58-85fc-9f351b1991d7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 09:18:53.929451 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:53.929355 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:18:53.929601 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:53.929364 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:53.929601 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:53.929494 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7trxt" podUID="67c71a8c-71f7-4494-a8cd-fd7b2af8ab51" Jun 15 09:18:53.929601 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:53.929562 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kddf5" podUID="56a0c215-5d6d-4cb2-b9aa-505333bc0cea" Jun 15 09:18:54.929251 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:54.929212 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:18:54.929665 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:54.929329 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdmf7" podUID="4e83c2a6-9354-4e58-85fc-9f351b1991d7" Jun 15 09:18:55.930405 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:55.930374 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:18:55.930900 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:55.930490 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7trxt" podUID="67c71a8c-71f7-4494-a8cd-fd7b2af8ab51" Jun 15 09:18:55.930900 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:55.930523 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:55.930900 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:55.930592 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kddf5" podUID="56a0c215-5d6d-4cb2-b9aa-505333bc0cea" Jun 15 09:18:56.929473 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:56.929440 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:18:56.929701 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:56.929544 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdmf7" podUID="4e83c2a6-9354-4e58-85fc-9f351b1991d7" Jun 15 09:18:57.930516 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:57.930301 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:18:57.931058 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:57.930358 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:57.931058 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:57.930561 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7trxt" podUID="67c71a8c-71f7-4494-a8cd-fd7b2af8ab51" Jun 15 09:18:57.931058 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:57.930644 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kddf5" podUID="56a0c215-5d6d-4cb2-b9aa-505333bc0cea" Jun 15 09:18:58.041687 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:58.041554 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pxdxj_f344540f-2677-49dd-8a0f-106be00f414d/ovn-acl-logging/0.log" Jun 15 09:18:58.042004 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:58.041981 2562 generic.go:356] "Generic (PLEG): container finished" podID="f344540f-2677-49dd-8a0f-106be00f414d" containerID="948718963b0bc2be7e5885f2d2a33072df1e1e9d7ec0e7b318820a88a9dc42d2" exitCode=1 Jun 15 09:18:58.042066 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:58.042052 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" event={"ID":"f344540f-2677-49dd-8a0f-106be00f414d","Type":"ContainerStarted","Data":"e9c4698b4601357569ac6ea2a95bc43c3ff202be25fccd8def1afc47dc543ca0"} Jun 15 09:18:58.042136 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:58.042123 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" event={"ID":"f344540f-2677-49dd-8a0f-106be00f414d","Type":"ContainerDied","Data":"948718963b0bc2be7e5885f2d2a33072df1e1e9d7ec0e7b318820a88a9dc42d2"} Jun 15 09:18:58.042174 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:58.042160 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" event={"ID":"f344540f-2677-49dd-8a0f-106be00f414d","Type":"ContainerStarted","Data":"56476cc6235094f4870af41b444e8fb91fdfecf991dce82c7385521800258734"} Jun 15 09:18:58.043259 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:58.043239 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-47q6n" event={"ID":"dd2e6f0c-a1ba-4f06-8286-ff69fdf52440","Type":"ContainerStarted","Data":"3aed72d82e1405cc779dfd90de9b10531fd8110f3ace472cb897b35ede10b091"} Jun 15 09:18:58.044515 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:58.044493 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" event={"ID":"449bde46-c22c-4d96-9b35-52c9130e47df","Type":"ContainerStarted","Data":"ecdacd27f9bc7b682a224e7fe0bf493a84c73905543ac2a0956e628408a0a6cc"} Jun 15 09:18:58.045720 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:58.045702 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sxdtv" event={"ID":"2699a480-5720-4d12-ab42-7c30a489e7d7","Type":"ContainerStarted","Data":"1943fa32980b67da1166d5fef4e6eca61f049cca98f322a87418778419a4bf85"} Jun 15 09:18:58.046872 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:58.046852 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jlhbm" event={"ID":"eac64cde-28f4-49be-8543-42e3b4ed898f","Type":"ContainerStarted","Data":"03aa181c5f0705a01611c40de3ecca016f06a72a8a8dbd6a53f497e2c1d8e35d"} Jun 15 09:18:58.048099 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:58.048082 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" event={"ID":"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f","Type":"ContainerStarted","Data":"f7157d67dc1a8ff0e992a1325b298b3885812f3752fc603348f096d966e9d189"} Jun 15 09:18:58.049372 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:58.049353 2562 generic.go:356] "Generic (PLEG): container finished" podID="92f911d1-972a-4024-9cd7-311529a59fb7" containerID="ba99c037ab2da972ae1d21babe4a0ed4aa4f5c8b3af512302fe54ce10b6193bd" exitCode=0 Jun 15 09:18:58.049445 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:58.049414 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44gsl" event={"ID":"92f911d1-972a-4024-9cd7-311529a59fb7","Type":"ContainerDied","Data":"ba99c037ab2da972ae1d21babe4a0ed4aa4f5c8b3af512302fe54ce10b6193bd"} Jun 15 09:18:58.050756 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:58.050668 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lfzxm" event={"ID":"a11b7428-db5b-4672-97f0-4cfaf32de873","Type":"ContainerStarted","Data":"2d9b35ba093efe4d136eaa4724dfb84f45a6163bffe04ed728b57de84097ad1a"} Jun 15 09:18:58.070028 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:58.069990 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-143.ec2.internal" podStartSLOduration=21.069978652 podStartE2EDuration="21.069978652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 09:18:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:18:41.016333115 +0000 UTC m=+5.579755423" watchObservedRunningTime="2026-06-15 09:18:58.069978652 +0000 UTC m=+22.633400956" Jun 15 09:18:58.081687 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:58.081648 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-47q6n" podStartSLOduration=3.553490483 podStartE2EDuration="22.081633489s" totalImagesPullingTime="18.528143006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:18:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:18:58.081268178 +0000 UTC m=+22.644690503" watchObservedRunningTime="2026-06-15 09:18:58.081633489 +0000 UTC m=+22.645055794" Jun 15 09:18:58.094947 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:58.094912 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jlhbm" podStartSLOduration=3.534577734 podStartE2EDuration="22.094901802s" totalImagesPullingTime="18.560324068s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 09:18:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:18:58.094402476 +0000 UTC m=+22.657824941" watchObservedRunningTime="2026-06-15 09:18:58.094901802 +0000 UTC m=+22.658324107" Jun 15 09:18:58.107737 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:58.107691 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lfzxm" podStartSLOduration=3.5036017900000003 podStartE2EDuration="22.107679445s" totalImagesPullingTime="18.604077655s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 09:18:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:18:58.10737631 +0000 UTC m=+22.670798616" watchObservedRunningTime="2026-06-15 09:18:58.107679445 +0000 UTC m=+22.671101750" Jun 15 09:18:58.126312 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:58.126275 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bkgdl" podStartSLOduration=3.522704189 podStartE2EDuration="22.126265565s" totalImagesPullingTime="18.603561376s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 09:18:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:18:58.125864103 +0000 UTC m=+22.689286408" watchObservedRunningTime="2026-06-15 09:18:58.126265565 +0000 UTC m=+22.689687870" Jun 15 09:18:58.141279 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:58.141249 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-sxdtv" podStartSLOduration=3.518927435 podStartE2EDuration="22.141238132s" totalImagesPullingTime="18.622310697s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 09:18:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:18:58.140968437 +0000 UTC m=+22.704390774" watchObservedRunningTime="2026-06-15 09:18:58.141238132 +0000 UTC m=+22.704660436" Jun 15 09:18:58.929428 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:58.929394 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:18:58.929582 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:58.929513 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdmf7" podUID="4e83c2a6-9354-4e58-85fc-9f351b1991d7" Jun 15 09:18:59.055093 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:59.055068 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pxdxj_f344540f-2677-49dd-8a0f-106be00f414d/ovn-acl-logging/0.log" Jun 15 09:18:59.055679 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:59.055450 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" event={"ID":"f344540f-2677-49dd-8a0f-106be00f414d","Type":"ContainerStarted","Data":"f41fff5ac9ae3625b6d09731d624125957c23f7bf7b2cae3f0e68ed297c2b890"} Jun 15 09:18:59.055679 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:59.055521 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" event={"ID":"f344540f-2677-49dd-8a0f-106be00f414d","Type":"ContainerStarted","Data":"85a5ab7ce84827d0823c93a554ef448c33f4973d4f9eae78069cfbac28a4d76a"} Jun 15 09:18:59.055679 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:59.055565 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" event={"ID":"f344540f-2677-49dd-8a0f-106be00f414d","Type":"ContainerStarted","Data":"3a3aa1967a800b1946f1516d00a571f41a268b285186908b62de0745da505d3f"} Jun 15 09:18:59.056865 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:59.056781 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p6nnr" event={"ID":"d7a4b513-5f5a-49ca-b112-6c4c00223f04","Type":"ContainerStarted","Data":"21ea6d4c559db2e032e80e3792f4b43a9c53a230e4ae6352a4df5ab2b9ad07c8"} Jun 15 09:18:59.070743 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:59.070705 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-p6nnr" podStartSLOduration=4.458815555 podStartE2EDuration="23.070691602s" totalImagesPullingTime="18.611876047s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 09:18:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:18:59.070229081 +0000 UTC m=+23.633651404" watchObservedRunningTime="2026-06-15 09:18:59.070691602 +0000 UTC m=+23.634113906" Jun 15 09:18:59.199406 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:59.199383 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 15 09:18:59.930293 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:59.930268 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:18:59.930607 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:59.930324 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:18:59.930607 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:59.930408 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kddf5" podUID="56a0c215-5d6d-4cb2-b9aa-505333bc0cea" Jun 15 09:18:59.930607 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:59.930377 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-15T09:18:59.199401583Z","UUID":"53148d1b-3d40-4640-b113-eed90b5662f1","Handler":null,"Name":"","Endpoint":""} Jun 15 09:18:59.930607 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:18:59.930586 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7trxt" podUID="67c71a8c-71f7-4494-a8cd-fd7b2af8ab51" Jun 15 09:18:59.932096 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:59.932077 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 15 09:18:59.932232 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:18:59.932114 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 15 09:19:00.063720 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:00.063673 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" event={"ID":"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f","Type":"ContainerStarted","Data":"bcefb259bf416d44fffd09e8f17cf88c57bf49585b171e0cc1818d6c717b50e0"} Jun 15 09:19:00.929485 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:00.929326 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:19:00.929724 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:00.929551 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdmf7" podUID="4e83c2a6-9354-4e58-85fc-9f351b1991d7" Jun 15 09:19:01.070035 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:01.070005 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pxdxj_f344540f-2677-49dd-8a0f-106be00f414d/ovn-acl-logging/0.log" Jun 15 09:19:01.070472 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:01.070418 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" event={"ID":"f344540f-2677-49dd-8a0f-106be00f414d","Type":"ContainerStarted","Data":"a619762381cd70959bc60c828751047bc05235de868b4e16f835618be87abe59"} Jun 15 09:19:01.072819 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:01.072789 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" event={"ID":"facd9737-c7d9-4fa5-9ece-f598f2c0cf1f","Type":"ContainerStarted","Data":"b9b581fe6907cd7b6684e584f96c91d0df1a8a2c61a9fcacb435fcb64575738b"} Jun 15 09:19:01.929738 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:01.929708 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:19:01.929920 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:01.929817 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7trxt" podUID="67c71a8c-71f7-4494-a8cd-fd7b2af8ab51" Jun 15 09:19:01.929920 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:01.929877 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:19:01.929997 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:01.929958 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kddf5" podUID="56a0c215-5d6d-4cb2-b9aa-505333bc0cea" Jun 15 09:19:02.442533 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:02.442468 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-47q6n" Jun 15 09:19:02.443116 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:02.443092 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-47q6n" Jun 15 09:19:02.480384 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:02.480335 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k58cq" podStartSLOduration=5.614214505 podStartE2EDuration="26.480317608s" totalImagesPullingTime="20.866103103s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 09:18:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:19:01.088800053 +0000 UTC m=+25.652222384" watchObservedRunningTime="2026-06-15 09:19:02.480317608 +0000 UTC m=+27.043739916" Jun 15 09:19:02.929615 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:02.929584 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:19:02.929809 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:02.929716 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdmf7" podUID="4e83c2a6-9354-4e58-85fc-9f351b1991d7" Jun 15 09:19:03.930527 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:03.930367 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:19:03.931010 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:03.930410 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:19:03.931010 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:03.930627 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7trxt" podUID="67c71a8c-71f7-4494-a8cd-fd7b2af8ab51" Jun 15 09:19:03.931010 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:03.930665 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kddf5" podUID="56a0c215-5d6d-4cb2-b9aa-505333bc0cea" Jun 15 09:19:04.082625 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:04.082594 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pxdxj_f344540f-2677-49dd-8a0f-106be00f414d/ovn-acl-logging/0.log" Jun 15 09:19:04.083007 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:04.082976 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" event={"ID":"f344540f-2677-49dd-8a0f-106be00f414d","Type":"ContainerStarted","Data":"e9c387e20c1678ebf46cd13f5cc4f87280b0ae56cc1ff545a5f6a2f8de9752a5"} Jun 15 09:19:04.083254 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:04.083231 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:19:04.083372 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:04.083303 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:19:04.083372 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:04.083350 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:19:04.083473 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:04.083448 2562 scope.go:117] "RemoveContainer" containerID="948718963b0bc2be7e5885f2d2a33072df1e1e9d7ec0e7b318820a88a9dc42d2" Jun 15 09:19:04.085628 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:04.085243 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44gsl" event={"ID":"92f911d1-972a-4024-9cd7-311529a59fb7","Type":"ContainerStarted","Data":"860967036c155edc0a5c2c261d35dc1c3d6eb6ec76b25ab6419d529060cafbbe"} Jun 15 09:19:04.099094 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:04.099073 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:19:04.099560 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:04.099541 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:19:04.616712 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:04.616681 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:04.696109 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:04.696077 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:04.929484 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:04.929395 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:19:04.929726 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:04.929522 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdmf7" podUID="4e83c2a6-9354-4e58-85fc-9f351b1991d7" Jun 15 09:19:05.091316 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:05.091290 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pxdxj_f344540f-2677-49dd-8a0f-106be00f414d/ovn-acl-logging/0.log" Jun 15 09:19:05.091713 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:05.091667 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" event={"ID":"f344540f-2677-49dd-8a0f-106be00f414d","Type":"ContainerStarted","Data":"cfd256b58c98c45f8dfe0517280ca2d8d86b8aab8a455b9235f6ec2b7a15e472"} Jun 15 09:19:05.093402 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:05.093381 2562 generic.go:356] "Generic (PLEG): container finished" podID="92f911d1-972a-4024-9cd7-311529a59fb7" containerID="860967036c155edc0a5c2c261d35dc1c3d6eb6ec76b25ab6419d529060cafbbe" exitCode=0 Jun 15 09:19:05.093535 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:05.093415 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44gsl" event={"ID":"92f911d1-972a-4024-9cd7-311529a59fb7","Type":"ContainerDied","Data":"860967036c155edc0a5c2c261d35dc1c3d6eb6ec76b25ab6419d529060cafbbe"} Jun 15 09:19:05.119325 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:05.119280 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" podStartSLOduration=10.306472288 podStartE2EDuration="29.119266583s" totalImagesPullingTime="18.812794295s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 09:18:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:19:05.11777595 +0000 UTC m=+29.681198277" watchObservedRunningTime="2026-06-15 09:19:05.119266583 +0000 UTC m=+29.682688914" Jun 15 09:19:05.313407 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:05.313311 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:05.413732 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:05.413697 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:05.644522 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:05.644492 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kddf5"] Jun 15 09:19:05.644703 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:05.644650 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:19:05.644797 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:05.644776 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kddf5" podUID="56a0c215-5d6d-4cb2-b9aa-505333bc0cea" Jun 15 09:19:05.647265 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:05.647241 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cdmf7"] Jun 15 09:19:05.647390 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:05.647345 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:19:05.647463 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:05.647442 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdmf7" podUID="4e83c2a6-9354-4e58-85fc-9f351b1991d7" Jun 15 09:19:05.647818 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:05.647797 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7trxt"] Jun 15 09:19:05.647945 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:05.647910 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:19:05.648008 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:05.647994 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7trxt" podUID="67c71a8c-71f7-4494-a8cd-fd7b2af8ab51" Jun 15 09:19:06.211533 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:06.211378 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-47q6n" Jun 15 09:19:06.211877 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:06.211625 2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 15 09:19:06.211955 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:06.211876 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-47q6n" Jun 15 09:19:06.273246 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:06.273214 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:06.930038 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:06.929965 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:19:06.930184 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:06.930072 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:19:06.930184 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:06.930072 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:19:06.930184 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:06.930151 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kddf5" podUID="56a0c215-5d6d-4cb2-b9aa-505333bc0cea" Jun 15 09:19:06.930330 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:06.930064 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdmf7" podUID="4e83c2a6-9354-4e58-85fc-9f351b1991d7" Jun 15 09:19:06.930330 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:06.930249 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7trxt" podUID="67c71a8c-71f7-4494-a8cd-fd7b2af8ab51" Jun 15 09:19:06.985178 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:06.985147 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:07.084494 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:07.084461 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:07.100475 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:07.100442 2562 generic.go:356] "Generic (PLEG): container finished" podID="92f911d1-972a-4024-9cd7-311529a59fb7" containerID="2c33605792b67ab59fa088e0fb78574a0f99323ff471bccc2777c46fd6c5503f" exitCode=0 Jun 15 09:19:07.100618 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:07.100514 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44gsl" event={"ID":"92f911d1-972a-4024-9cd7-311529a59fb7","Type":"ContainerDied","Data":"2c33605792b67ab59fa088e0fb78574a0f99323ff471bccc2777c46fd6c5503f"} Jun 15 09:19:07.137071 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:07.137043 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:08.001327 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:08.001240 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:08.101201 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:08.101163 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:08.397493 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:08.397461 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:08.929643 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:08.929611 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:19:08.929829 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:08.929611 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:19:08.929829 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:08.929725 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kddf5" podUID="56a0c215-5d6d-4cb2-b9aa-505333bc0cea" Jun 15 09:19:08.929829 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:08.929614 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:19:08.929984 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:08.929819 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7trxt" podUID="67c71a8c-71f7-4494-a8cd-fd7b2af8ab51" Jun 15 09:19:08.929984 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:08.929871 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdmf7" podUID="4e83c2a6-9354-4e58-85fc-9f351b1991d7" Jun 15 09:19:08.933033 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:08.933006 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:09.106423 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:09.106391 2562 generic.go:356] "Generic (PLEG): container finished" podID="92f911d1-972a-4024-9cd7-311529a59fb7" containerID="c15884e0b2c71a5f36b4e9805f64725093f3753d2cbdad1e7325278160f066aa" exitCode=0 Jun 15 09:19:09.106882 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:09.106451 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44gsl" event={"ID":"92f911d1-972a-4024-9cd7-311529a59fb7","Type":"ContainerDied","Data":"c15884e0b2c71a5f36b4e9805f64725093f3753d2cbdad1e7325278160f066aa"} Jun 15 09:19:09.628022 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:09.627989 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs\") pod \"network-metrics-daemon-7trxt\" (UID: \"67c71a8c-71f7-4494-a8cd-fd7b2af8ab51\") " pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:19:09.628184 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:09.628118 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 09:19:09.628184 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:09.628169 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs podName:67c71a8c-71f7-4494-a8cd-fd7b2af8ab51 nodeName:}" failed. No retries permitted until 2026-06-15 09:19:41.628155745 +0000 UTC m=+66.191578028 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs") pod "network-metrics-daemon-7trxt" (UID: "67c71a8c-71f7-4494-a8cd-fd7b2af8ab51") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 09:19:09.729066 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:09.729036 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret\") pod \"global-pull-secret-syncer-kddf5\" (UID: \"56a0c215-5d6d-4cb2-b9aa-505333bc0cea\") " pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:19:09.729231 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:09.729168 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 09:19:09.729286 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:09.729236 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret podName:56a0c215-5d6d-4cb2-b9aa-505333bc0cea nodeName:}" failed. No retries permitted until 2026-06-15 09:19:41.729216489 +0000 UTC m=+66.292638784 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret") pod "global-pull-secret-syncer-kddf5" (UID: "56a0c215-5d6d-4cb2-b9aa-505333bc0cea") : object "kube-system"/"original-pull-secret" not registered Jun 15 09:19:09.829348 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:09.829317 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2vs8\" (UniqueName: \"kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8\") pod \"network-check-target-cdmf7\" (UID: \"4e83c2a6-9354-4e58-85fc-9f351b1991d7\") " pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:19:09.829483 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:09.829471 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 09:19:09.829555 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:09.829488 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 09:19:09.829555 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:09.829499 2562 projected.go:196] Error preparing data for projected volume kube-api-access-x2vs8 for pod openshift-network-diagnostics/network-check-target-cdmf7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 09:19:09.829555 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:09.829551 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8 podName:4e83c2a6-9354-4e58-85fc-9f351b1991d7 nodeName:}" failed. No retries permitted until 2026-06-15 09:19:41.829534123 +0000 UTC m=+66.392956428 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-x2vs8" (UniqueName: "kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8") pod "network-check-target-cdmf7" (UID: "4e83c2a6-9354-4e58-85fc-9f351b1991d7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 09:19:10.167339 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.167313 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-143.ec2.internal" event="NodeReady" Jun 15 09:19:10.167848 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.167426 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 15 09:19:10.213075 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.213045 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rr6xd"] Jun 15 09:19:10.214983 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.214963 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-66c98d86-pkr99"] Jun 15 09:19:10.215126 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.215107 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rr6xd" Jun 15 09:19:10.216380 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.216361 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gwnsl"] Jun 15 09:19:10.216526 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.216503 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.217206 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.217186 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 15 09:19:10.217328 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.217306 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g8xgb\"" Jun 15 09:19:10.217390 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.217306 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 15 09:19:10.218505 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.218488 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 15 09:19:10.218723 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.218709 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gwnsl" Jun 15 09:19:10.218806 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.218734 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 15 09:19:10.219221 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.219181 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ksdhw\"" Jun 15 09:19:10.219221 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.219203 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 15 09:19:10.221223 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.221206 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 15 09:19:10.221911 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.221722 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 15 09:19:10.221911 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.221791 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rfkqb\"" Jun 15 09:19:10.221911 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.221823 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 15 09:19:10.226260 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.225483 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 15 09:19:10.226688 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.226572 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gwnsl"] Jun 15 09:19:10.227412 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.227370 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rr6xd"] Jun 15 09:19:10.244719 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.244697 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66c98d86-pkr99"] Jun 15 09:19:10.319849 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.319805 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:10.332767 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.332735 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-installation-pull-secrets\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.332930 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.332777 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-bound-sa-token\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.332930 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.332855 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-ca-trust-extracted\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.332930 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.332880 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj8jf\" (UniqueName: \"kubernetes.io/projected/695a033a-993a-4bd2-9f73-3f979df52254-kube-api-access-qj8jf\") pod \"dns-default-rr6xd\" (UID: \"695a033a-993a-4bd2-9f73-3f979df52254\") " pod="openshift-dns/dns-default-rr6xd" Jun 15 09:19:10.332930 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.332910 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-trusted-ca\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.333102 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.332938 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt5mz\" (UniqueName: \"kubernetes.io/projected/22891e8c-12f1-48c2-910f-5015573cdad8-kube-api-access-wt5mz\") pod \"ingress-canary-gwnsl\" (UID: \"22891e8c-12f1-48c2-910f-5015573cdad8\") " pod="openshift-ingress-canary/ingress-canary-gwnsl" Jun 15 09:19:10.333102 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.333020 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22891e8c-12f1-48c2-910f-5015573cdad8-cert\") pod \"ingress-canary-gwnsl\" (UID: \"22891e8c-12f1-48c2-910f-5015573cdad8\") " pod="openshift-ingress-canary/ingress-canary-gwnsl" Jun 15 09:19:10.333102 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.333072 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-tls\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.333222 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.333112 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-image-registry-private-configuration\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.333222 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.333135 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/695a033a-993a-4bd2-9f73-3f979df52254-tmp-dir\") pod \"dns-default-rr6xd\" (UID: \"695a033a-993a-4bd2-9f73-3f979df52254\") " pod="openshift-dns/dns-default-rr6xd" Jun 15 09:19:10.333222 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.333166 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmxw7\" (UniqueName: \"kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-kube-api-access-hmxw7\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.333222 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.333186 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/695a033a-993a-4bd2-9f73-3f979df52254-config-volume\") pod \"dns-default-rr6xd\" (UID: \"695a033a-993a-4bd2-9f73-3f979df52254\") " pod="openshift-dns/dns-default-rr6xd" Jun 15 09:19:10.333222 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.333207 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/695a033a-993a-4bd2-9f73-3f979df52254-metrics-tls\") pod \"dns-default-rr6xd\" (UID: \"695a033a-993a-4bd2-9f73-3f979df52254\") " pod="openshift-dns/dns-default-rr6xd" Jun 15 09:19:10.333427 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.333271 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-certificates\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.434525 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.434491 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22891e8c-12f1-48c2-910f-5015573cdad8-cert\") pod \"ingress-canary-gwnsl\" (UID: \"22891e8c-12f1-48c2-910f-5015573cdad8\") " pod="openshift-ingress-canary/ingress-canary-gwnsl" Jun 15 09:19:10.434525 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.434526 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-tls\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.434721 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.434547 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-image-registry-private-configuration\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.434721 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:10.434645 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 09:19:10.434721 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:10.434659 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66c98d86-pkr99: secret "image-registry-tls" not found Jun 15 09:19:10.434898 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:10.434726 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-tls podName:0690cee7-ba7d-4990-9e23-4d86f39cdd5c nodeName:}" failed. No retries permitted until 2026-06-15 09:19:10.93470791 +0000 UTC m=+35.498130211 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-tls") pod "image-registry-66c98d86-pkr99" (UID: "0690cee7-ba7d-4990-9e23-4d86f39cdd5c") : secret "image-registry-tls" not found Jun 15 09:19:10.435017 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.434658 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/695a033a-993a-4bd2-9f73-3f979df52254-tmp-dir\") pod \"dns-default-rr6xd\" (UID: \"695a033a-993a-4bd2-9f73-3f979df52254\") " pod="openshift-dns/dns-default-rr6xd" Jun 15 09:19:10.435135 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.435018 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/695a033a-993a-4bd2-9f73-3f979df52254-tmp-dir\") pod \"dns-default-rr6xd\" (UID: \"695a033a-993a-4bd2-9f73-3f979df52254\") " pod="openshift-dns/dns-default-rr6xd" Jun 15 09:19:10.435135 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.435022 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmxw7\" (UniqueName: \"kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-kube-api-access-hmxw7\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.435135 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.435071 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/695a033a-993a-4bd2-9f73-3f979df52254-config-volume\") pod \"dns-default-rr6xd\" (UID: \"695a033a-993a-4bd2-9f73-3f979df52254\") " pod="openshift-dns/dns-default-rr6xd" Jun 15 09:19:10.435135 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.435092 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/695a033a-993a-4bd2-9f73-3f979df52254-metrics-tls\") pod \"dns-default-rr6xd\" (UID: \"695a033a-993a-4bd2-9f73-3f979df52254\") " pod="openshift-dns/dns-default-rr6xd" Jun 15 09:19:10.435135 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.435122 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-certificates\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.435396 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.435155 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-installation-pull-secrets\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.435396 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.435182 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-bound-sa-token\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.435396 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.435234 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-ca-trust-extracted\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.435396 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.435258 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj8jf\" (UniqueName: \"kubernetes.io/projected/695a033a-993a-4bd2-9f73-3f979df52254-kube-api-access-qj8jf\") pod \"dns-default-rr6xd\" (UID: \"695a033a-993a-4bd2-9f73-3f979df52254\") " pod="openshift-dns/dns-default-rr6xd" Jun 15 09:19:10.435766 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.435745 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/695a033a-993a-4bd2-9f73-3f979df52254-config-volume\") pod \"dns-default-rr6xd\" (UID: \"695a033a-993a-4bd2-9f73-3f979df52254\") " pod="openshift-dns/dns-default-rr6xd" Jun 15 09:19:10.435885 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.435865 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-ca-trust-extracted\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.435949 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:10.435934 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 09:19:10.435981 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.435967 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-certificates\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.436029 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:10.435989 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/695a033a-993a-4bd2-9f73-3f979df52254-metrics-tls podName:695a033a-993a-4bd2-9f73-3f979df52254 nodeName:}" failed. No retries permitted until 2026-06-15 09:19:10.935971589 +0000 UTC m=+35.499393886 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/695a033a-993a-4bd2-9f73-3f979df52254-metrics-tls") pod "dns-default-rr6xd" (UID: "695a033a-993a-4bd2-9f73-3f979df52254") : secret "dns-default-metrics-tls" not found Jun 15 09:19:10.436029 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.436012 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-trusted-ca\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.436133 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.436050 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt5mz\" (UniqueName: \"kubernetes.io/projected/22891e8c-12f1-48c2-910f-5015573cdad8-kube-api-access-wt5mz\") pod \"ingress-canary-gwnsl\" (UID: \"22891e8c-12f1-48c2-910f-5015573cdad8\") " pod="openshift-ingress-canary/ingress-canary-gwnsl" Jun 15 09:19:10.436876 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.436856 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-trusted-ca\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.439536 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.439514 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-image-registry-private-configuration\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.439640 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.439518 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22891e8c-12f1-48c2-910f-5015573cdad8-cert\") pod \"ingress-canary-gwnsl\" (UID: \"22891e8c-12f1-48c2-910f-5015573cdad8\") " pod="openshift-ingress-canary/ingress-canary-gwnsl" Jun 15 09:19:10.439640 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.439564 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-installation-pull-secrets\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.445343 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.444500 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj8jf\" (UniqueName: \"kubernetes.io/projected/695a033a-993a-4bd2-9f73-3f979df52254-kube-api-access-qj8jf\") pod \"dns-default-rr6xd\" (UID: \"695a033a-993a-4bd2-9f73-3f979df52254\") " pod="openshift-dns/dns-default-rr6xd" Jun 15 09:19:10.445343 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.444691 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt5mz\" (UniqueName: \"kubernetes.io/projected/22891e8c-12f1-48c2-910f-5015573cdad8-kube-api-access-wt5mz\") pod \"ingress-canary-gwnsl\" (UID: \"22891e8c-12f1-48c2-910f-5015573cdad8\") " pod="openshift-ingress-canary/ingress-canary-gwnsl" Jun 15 09:19:10.445343 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.444750 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmxw7\" (UniqueName: \"kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-kube-api-access-hmxw7\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.445343 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.445336 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-bound-sa-token\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.542538 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.542504 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gwnsl" Jun 15 09:19:10.691977 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.691778 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gwnsl"] Jun 15 09:19:10.697757 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:19:10.697733 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22891e8c_12f1_48c2_910f_5015573cdad8.slice/crio-22df5c1272d9cc130fd0620b6240e707a9728b9c0e353707d61b57f28f57b3f7 WatchSource:0}: Error finding container 22df5c1272d9cc130fd0620b6240e707a9728b9c0e353707d61b57f28f57b3f7: Status 404 returned error can't find the container with id 22df5c1272d9cc130fd0620b6240e707a9728b9c0e353707d61b57f28f57b3f7 Jun 15 09:19:10.929604 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.929497 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:19:10.929604 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.929524 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:19:10.929604 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.929524 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:19:10.932304 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.932276 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 15 09:19:10.933276 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.933256 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vhk77\"" Jun 15 09:19:10.933394 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.933335 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 15 09:19:10.933464 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.933255 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hsb9v\"" Jun 15 09:19:10.933519 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.933254 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 15 09:19:10.933610 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.933589 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 15 09:19:10.940892 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.940857 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-tls\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:10.940998 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:10.940909 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/695a033a-993a-4bd2-9f73-3f979df52254-metrics-tls\") pod \"dns-default-rr6xd\" (UID: \"695a033a-993a-4bd2-9f73-3f979df52254\") " pod="openshift-dns/dns-default-rr6xd" Jun 15 09:19:10.941048 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:10.941002 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 09:19:10.941096 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:10.941066 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/695a033a-993a-4bd2-9f73-3f979df52254-metrics-tls podName:695a033a-993a-4bd2-9f73-3f979df52254 nodeName:}" failed. No retries permitted until 2026-06-15 09:19:11.941046451 +0000 UTC m=+36.504468734 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/695a033a-993a-4bd2-9f73-3f979df52254-metrics-tls") pod "dns-default-rr6xd" (UID: "695a033a-993a-4bd2-9f73-3f979df52254") : secret "dns-default-metrics-tls" not found Jun 15 09:19:10.941162 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:10.941143 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 09:19:10.941217 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:10.941165 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66c98d86-pkr99: secret "image-registry-tls" not found Jun 15 09:19:10.941264 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:10.941221 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-tls podName:0690cee7-ba7d-4990-9e23-4d86f39cdd5c nodeName:}" failed. No retries permitted until 2026-06-15 09:19:11.941201468 +0000 UTC m=+36.504623754 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-tls") pod "image-registry-66c98d86-pkr99" (UID: "0690cee7-ba7d-4990-9e23-4d86f39cdd5c") : secret "image-registry-tls" not found Jun 15 09:19:11.112706 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:11.112667 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gwnsl" event={"ID":"22891e8c-12f1-48c2-910f-5015573cdad8","Type":"ContainerStarted","Data":"22df5c1272d9cc130fd0620b6240e707a9728b9c0e353707d61b57f28f57b3f7"} Jun 15 09:19:11.948401 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:11.948365 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/695a033a-993a-4bd2-9f73-3f979df52254-metrics-tls\") pod \"dns-default-rr6xd\" (UID: \"695a033a-993a-4bd2-9f73-3f979df52254\") " pod="openshift-dns/dns-default-rr6xd" Jun 15 09:19:11.948911 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:11.948496 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-tls\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:11.948911 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:11.948529 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 09:19:11.948911 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:11.948596 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/695a033a-993a-4bd2-9f73-3f979df52254-metrics-tls podName:695a033a-993a-4bd2-9f73-3f979df52254 nodeName:}" failed. No retries permitted until 2026-06-15 09:19:13.948576345 +0000 UTC m=+38.511998636 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/695a033a-993a-4bd2-9f73-3f979df52254-metrics-tls") pod "dns-default-rr6xd" (UID: "695a033a-993a-4bd2-9f73-3f979df52254") : secret "dns-default-metrics-tls" not found Jun 15 09:19:11.948911 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:11.948615 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 09:19:11.948911 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:11.948629 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66c98d86-pkr99: secret "image-registry-tls" not found Jun 15 09:19:11.948911 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:11.948677 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-tls podName:0690cee7-ba7d-4990-9e23-4d86f39cdd5c nodeName:}" failed. No retries permitted until 2026-06-15 09:19:13.948661049 +0000 UTC m=+38.512083354 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-tls") pod "image-registry-66c98d86-pkr99" (UID: "0690cee7-ba7d-4990-9e23-4d86f39cdd5c") : secret "image-registry-tls" not found Jun 15 09:19:13.119661 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:13.119619 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gwnsl" event={"ID":"22891e8c-12f1-48c2-910f-5015573cdad8","Type":"ContainerStarted","Data":"389e82763c438dc24ed96c23cdf17010fc5cc720ac9e931613824e8ae5eba6b8"} Jun 15 09:19:13.135446 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:13.135395 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gwnsl" podStartSLOduration=1.4888440840000001 podStartE2EDuration="3.135376471s" totalImagesPullingTime="1.646532387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:19:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:19:13.134071895 +0000 UTC m=+37.697494218" watchObservedRunningTime="2026-06-15 09:19:13.135376471 +0000 UTC m=+37.698798778" Jun 15 09:19:13.964672 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:13.964640 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-tls\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:13.964672 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:13.964678 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/695a033a-993a-4bd2-9f73-3f979df52254-metrics-tls\") pod \"dns-default-rr6xd\" (UID: \"695a033a-993a-4bd2-9f73-3f979df52254\") " pod="openshift-dns/dns-default-rr6xd" Jun 15 09:19:13.964947 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:13.964781 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 09:19:13.964947 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:13.964788 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 09:19:13.964947 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:13.964807 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66c98d86-pkr99: secret "image-registry-tls" not found Jun 15 09:19:13.964947 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:13.964867 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/695a033a-993a-4bd2-9f73-3f979df52254-metrics-tls podName:695a033a-993a-4bd2-9f73-3f979df52254 nodeName:}" failed. No retries permitted until 2026-06-15 09:19:17.964830674 +0000 UTC m=+42.528252979 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/695a033a-993a-4bd2-9f73-3f979df52254-metrics-tls") pod "dns-default-rr6xd" (UID: "695a033a-993a-4bd2-9f73-3f979df52254") : secret "dns-default-metrics-tls" not found Jun 15 09:19:13.964947 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:13.964882 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-tls podName:0690cee7-ba7d-4990-9e23-4d86f39cdd5c nodeName:}" failed. No retries permitted until 2026-06-15 09:19:17.964875857 +0000 UTC m=+42.528298140 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-tls") pod "image-registry-66c98d86-pkr99" (UID: "0690cee7-ba7d-4990-9e23-4d86f39cdd5c") : secret "image-registry-tls" not found Jun 15 09:19:14.843970 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:14.843771 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:14.929325 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:14.929295 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:14.987701 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:14.987653 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:15.126399 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:15.126329 2562 generic.go:356] "Generic (PLEG): container finished" podID="92f911d1-972a-4024-9cd7-311529a59fb7" containerID="a8544e0fb93d478821bcc8620de5e34dfbccbf2b5e045b6375dc5ca45976fea2" exitCode=0 Jun 15 09:19:15.126399 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:15.126367 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44gsl" event={"ID":"92f911d1-972a-4024-9cd7-311529a59fb7","Type":"ContainerDied","Data":"a8544e0fb93d478821bcc8620de5e34dfbccbf2b5e045b6375dc5ca45976fea2"} Jun 15 09:19:15.851979 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:15.851937 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:15.858646 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:15.858619 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d4bcd4867-kdt7s"] Jun 15 09:19:15.882618 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:15.882596 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d4bcd4867-kdt7s"] Jun 15 09:19:15.882706 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:15.882682 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-kdt7s" Jun 15 09:19:15.885113 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:15.885085 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jun 15 09:19:15.885207 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:15.885135 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-rmm2q\"" Jun 15 09:19:15.886023 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:15.886006 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jun 15 09:19:15.945895 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:15.945827 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:15.983040 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:15.983010 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvtxs\" (UniqueName: \"kubernetes.io/projected/4bb39dda-ff30-41eb-a023-8866e8df5889-kube-api-access-kvtxs\") pod \"migrator-d4bcd4867-kdt7s\" (UID: \"4bb39dda-ff30-41eb-a023-8866e8df5889\") " pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-kdt7s" Jun 15 09:19:16.007604 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.007574 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:16.083421 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.083392 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kz4mb"] Jun 15 09:19:16.083542 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.083480 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvtxs\" (UniqueName: \"kubernetes.io/projected/4bb39dda-ff30-41eb-a023-8866e8df5889-kube-api-access-kvtxs\") pod \"migrator-d4bcd4867-kdt7s\" (UID: \"4bb39dda-ff30-41eb-a023-8866e8df5889\") " pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-kdt7s" Jun 15 09:19:16.095938 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.095912 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvtxs\" (UniqueName: \"kubernetes.io/projected/4bb39dda-ff30-41eb-a023-8866e8df5889-kube-api-access-kvtxs\") pod \"migrator-d4bcd4867-kdt7s\" (UID: \"4bb39dda-ff30-41eb-a023-8866e8df5889\") " pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-kdt7s" Jun 15 09:19:16.101423 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.101398 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:16.102223 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.102181 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kz4mb"] Jun 15 09:19:16.102295 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.102284 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:16.104543 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.104525 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 15 09:19:16.104654 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.104557 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 15 09:19:16.104654 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.104575 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 15 09:19:16.104654 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.104586 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 15 09:19:16.104813 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.104727 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4h4xm\"" Jun 15 09:19:16.131319 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.131294 2562 generic.go:356] "Generic (PLEG): container finished" podID="92f911d1-972a-4024-9cd7-311529a59fb7" containerID="163e1f3d1add235c354e3941b3fa51dec53f191b272b7763a0a030c5a85d55de" exitCode=0 Jun 15 09:19:16.131435 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.131332 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44gsl" event={"ID":"92f911d1-972a-4024-9cd7-311529a59fb7","Type":"ContainerDied","Data":"163e1f3d1add235c354e3941b3fa51dec53f191b272b7763a0a030c5a85d55de"} Jun 15 09:19:16.160557 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.160522 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:16.185078 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.185053 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79f2f4a8-307d-48e2-9c31-83bcd1636183-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kz4mb\" (UID: \"79f2f4a8-307d-48e2-9c31-83bcd1636183\") " pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:16.185198 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.185114 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/79f2f4a8-307d-48e2-9c31-83bcd1636183-data-volume\") pod \"insights-runtime-extractor-kz4mb\" (UID: \"79f2f4a8-307d-48e2-9c31-83bcd1636183\") " pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:16.185198 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.185167 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79f2f4a8-307d-48e2-9c31-83bcd1636183-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kz4mb\" (UID: \"79f2f4a8-307d-48e2-9c31-83bcd1636183\") " pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:16.185320 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.185293 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkz49\" (UniqueName: \"kubernetes.io/projected/79f2f4a8-307d-48e2-9c31-83bcd1636183-kube-api-access-xkz49\") pod \"insights-runtime-extractor-kz4mb\" (UID: \"79f2f4a8-307d-48e2-9c31-83bcd1636183\") " pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:16.185390 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.185358 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/79f2f4a8-307d-48e2-9c31-83bcd1636183-crio-socket\") pod \"insights-runtime-extractor-kz4mb\" (UID: \"79f2f4a8-307d-48e2-9c31-83bcd1636183\") " pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:16.191664 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.191646 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-kdt7s" Jun 15 09:19:16.286271 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.286240 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkz49\" (UniqueName: \"kubernetes.io/projected/79f2f4a8-307d-48e2-9c31-83bcd1636183-kube-api-access-xkz49\") pod \"insights-runtime-extractor-kz4mb\" (UID: \"79f2f4a8-307d-48e2-9c31-83bcd1636183\") " pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:16.286393 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.286289 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/79f2f4a8-307d-48e2-9c31-83bcd1636183-crio-socket\") pod \"insights-runtime-extractor-kz4mb\" (UID: \"79f2f4a8-307d-48e2-9c31-83bcd1636183\") " pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:16.286393 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.286353 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79f2f4a8-307d-48e2-9c31-83bcd1636183-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kz4mb\" (UID: \"79f2f4a8-307d-48e2-9c31-83bcd1636183\") " pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:16.286494 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.286410 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/79f2f4a8-307d-48e2-9c31-83bcd1636183-data-volume\") pod \"insights-runtime-extractor-kz4mb\" (UID: \"79f2f4a8-307d-48e2-9c31-83bcd1636183\") " pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:16.286494 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.286444 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79f2f4a8-307d-48e2-9c31-83bcd1636183-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kz4mb\" (UID: \"79f2f4a8-307d-48e2-9c31-83bcd1636183\") " pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:16.286586 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:16.286540 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 15 09:19:16.286629 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:16.286611 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/79f2f4a8-307d-48e2-9c31-83bcd1636183-kube-rbac-proxy-cm podName:79f2f4a8-307d-48e2-9c31-83bcd1636183 nodeName:}" failed. No retries permitted until 2026-06-15 09:19:16.786589525 +0000 UTC m=+41.350011828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/79f2f4a8-307d-48e2-9c31-83bcd1636183-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kz4mb" (UID: "79f2f4a8-307d-48e2-9c31-83bcd1636183") : configmap "kube-rbac-proxy" not found Jun 15 09:19:16.286880 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:16.286754 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 09:19:16.286880 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:16.286813 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f2f4a8-307d-48e2-9c31-83bcd1636183-insights-runtime-extractor-tls podName:79f2f4a8-307d-48e2-9c31-83bcd1636183 nodeName:}" failed. No retries permitted until 2026-06-15 09:19:16.786794644 +0000 UTC m=+41.350216941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79f2f4a8-307d-48e2-9c31-83bcd1636183-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kz4mb" (UID: "79f2f4a8-307d-48e2-9c31-83bcd1636183") : secret "insights-runtime-extractor-tls" not found Jun 15 09:19:16.287012 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.286915 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/79f2f4a8-307d-48e2-9c31-83bcd1636183-crio-socket\") pod \"insights-runtime-extractor-kz4mb\" (UID: \"79f2f4a8-307d-48e2-9c31-83bcd1636183\") " pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:16.287113 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.287095 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/79f2f4a8-307d-48e2-9c31-83bcd1636183-data-volume\") pod \"insights-runtime-extractor-kz4mb\" (UID: \"79f2f4a8-307d-48e2-9c31-83bcd1636183\") " pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:16.310238 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.310210 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkz49\" (UniqueName: \"kubernetes.io/projected/79f2f4a8-307d-48e2-9c31-83bcd1636183-kube-api-access-xkz49\") pod \"insights-runtime-extractor-kz4mb\" (UID: \"79f2f4a8-307d-48e2-9c31-83bcd1636183\") " pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:16.343540 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.343511 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d4bcd4867-kdt7s"] Jun 15 09:19:16.346488 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:19:16.346461 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bb39dda_ff30_41eb_a023_8866e8df5889.slice/crio-4499fa829e2347ef723fc7157498b9d98415fb47d431dfb71cdf8284217f4127 WatchSource:0}: Error finding container 4499fa829e2347ef723fc7157498b9d98415fb47d431dfb71cdf8284217f4127: Status 404 returned error can't find the container with id 4499fa829e2347ef723fc7157498b9d98415fb47d431dfb71cdf8284217f4127 Jun 15 09:19:16.376412 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.376328 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:16.430221 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.430192 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:16.605353 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.605320 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:16.707353 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.707261 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:16.790259 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.790230 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79f2f4a8-307d-48e2-9c31-83bcd1636183-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kz4mb\" (UID: \"79f2f4a8-307d-48e2-9c31-83bcd1636183\") " pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:16.790417 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:16.790277 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79f2f4a8-307d-48e2-9c31-83bcd1636183-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kz4mb\" (UID: \"79f2f4a8-307d-48e2-9c31-83bcd1636183\") " pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:16.790417 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:16.790369 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 09:19:16.790417 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:16.790403 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 15 09:19:16.790531 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:16.790420 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f2f4a8-307d-48e2-9c31-83bcd1636183-insights-runtime-extractor-tls podName:79f2f4a8-307d-48e2-9c31-83bcd1636183 nodeName:}" failed. No retries permitted until 2026-06-15 09:19:17.790405924 +0000 UTC m=+42.353828208 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79f2f4a8-307d-48e2-9c31-83bcd1636183-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kz4mb" (UID: "79f2f4a8-307d-48e2-9c31-83bcd1636183") : secret "insights-runtime-extractor-tls" not found Jun 15 09:19:16.790531 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:16.790458 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/79f2f4a8-307d-48e2-9c31-83bcd1636183-kube-rbac-proxy-cm podName:79f2f4a8-307d-48e2-9c31-83bcd1636183 nodeName:}" failed. No retries permitted until 2026-06-15 09:19:17.790442821 +0000 UTC m=+42.353865105 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/79f2f4a8-307d-48e2-9c31-83bcd1636183-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kz4mb" (UID: "79f2f4a8-307d-48e2-9c31-83bcd1636183") : configmap "kube-rbac-proxy" not found Jun 15 09:19:17.136759 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:17.136722 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44gsl" event={"ID":"92f911d1-972a-4024-9cd7-311529a59fb7","Type":"ContainerStarted","Data":"ef07d00fb9027d866f771f47187bd40c7c147137cfb25ae621262bbaebc80bc8"} Jun 15 09:19:17.137766 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:17.137746 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-kdt7s" event={"ID":"4bb39dda-ff30-41eb-a023-8866e8df5889","Type":"ContainerStarted","Data":"4499fa829e2347ef723fc7157498b9d98415fb47d431dfb71cdf8284217f4127"} Jun 15 09:19:17.158779 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:17.158740 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-44gsl" podStartSLOduration=9.329798712 podStartE2EDuration="41.15872641s" totalImagesPullingTime="31.828927698s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 09:18:36 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:19:17.157396149 +0000 UTC m=+41.720818455" watchObservedRunningTime="2026-06-15 09:19:17.15872641 +0000 UTC m=+41.722148714" Jun 15 09:19:17.279122 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:17.279087 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:17.332726 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:17.332689 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:17.796968 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:17.796934 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79f2f4a8-307d-48e2-9c31-83bcd1636183-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kz4mb\" (UID: \"79f2f4a8-307d-48e2-9c31-83bcd1636183\") " pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:17.797132 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:17.796990 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79f2f4a8-307d-48e2-9c31-83bcd1636183-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kz4mb\" (UID: \"79f2f4a8-307d-48e2-9c31-83bcd1636183\") " pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:17.797132 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:17.797079 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 15 09:19:17.797132 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:17.797080 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 09:19:17.797132 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:17.797131 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/79f2f4a8-307d-48e2-9c31-83bcd1636183-kube-rbac-proxy-cm podName:79f2f4a8-307d-48e2-9c31-83bcd1636183 nodeName:}" failed. No retries permitted until 2026-06-15 09:19:19.797116739 +0000 UTC m=+44.360539022 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/79f2f4a8-307d-48e2-9c31-83bcd1636183-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kz4mb" (UID: "79f2f4a8-307d-48e2-9c31-83bcd1636183") : configmap "kube-rbac-proxy" not found Jun 15 09:19:17.797261 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:17.797144 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f2f4a8-307d-48e2-9c31-83bcd1636183-insights-runtime-extractor-tls podName:79f2f4a8-307d-48e2-9c31-83bcd1636183 nodeName:}" failed. No retries permitted until 2026-06-15 09:19:19.797137745 +0000 UTC m=+44.360560028 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79f2f4a8-307d-48e2-9c31-83bcd1636183-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kz4mb" (UID: "79f2f4a8-307d-48e2-9c31-83bcd1636183") : secret "insights-runtime-extractor-tls" not found Jun 15 09:19:17.998760 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:17.998724 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-tls\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:17.998952 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:17.998788 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/695a033a-993a-4bd2-9f73-3f979df52254-metrics-tls\") pod \"dns-default-rr6xd\" (UID: \"695a033a-993a-4bd2-9f73-3f979df52254\") " pod="openshift-dns/dns-default-rr6xd" Jun 15 09:19:17.998952 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:17.998911 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 15 09:19:17.998952 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:17.998940 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66c98d86-pkr99: secret "image-registry-tls" not found Jun 15 09:19:17.999128 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:17.999001 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-tls podName:0690cee7-ba7d-4990-9e23-4d86f39cdd5c nodeName:}" failed. No retries permitted until 2026-06-15 09:19:25.998980378 +0000 UTC m=+50.562402680 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-tls") pod "image-registry-66c98d86-pkr99" (UID: "0690cee7-ba7d-4990-9e23-4d86f39cdd5c") : secret "image-registry-tls" not found Jun 15 09:19:17.999128 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:17.998914 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 09:19:17.999128 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:17.999084 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/695a033a-993a-4bd2-9f73-3f979df52254-metrics-tls podName:695a033a-993a-4bd2-9f73-3f979df52254 nodeName:}" failed. No retries permitted until 2026-06-15 09:19:25.999065718 +0000 UTC m=+50.562488014 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/695a033a-993a-4bd2-9f73-3f979df52254-metrics-tls") pod "dns-default-rr6xd" (UID: "695a033a-993a-4bd2-9f73-3f979df52254") : secret "dns-default-metrics-tls" not found Jun 15 09:19:18.201598 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:18.201571 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7b994bc59-n8tmk"] Jun 15 09:19:18.217687 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:18.217662 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7b994bc59-n8tmk"] Jun 15 09:19:18.217807 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:18.217740 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7b994bc59-n8tmk" Jun 15 09:19:18.220469 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:18.220448 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 15 09:19:18.220564 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:18.220454 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 15 09:19:18.221280 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:18.221254 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-mktrz\"" Jun 15 09:19:18.221280 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:18.221276 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 15 09:19:18.221447 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:18.221290 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 15 09:19:18.235468 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:18.235445 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:18.286743 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:18.286715 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:18.301098 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:18.301078 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c876f183-f50d-41bb-8d34-903ac8165eeb-signing-key\") pod \"service-ca-7b994bc59-n8tmk\" (UID: \"c876f183-f50d-41bb-8d34-903ac8165eeb\") " pod="openshift-service-ca/service-ca-7b994bc59-n8tmk" Jun 15 09:19:18.301171 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:18.301103 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c876f183-f50d-41bb-8d34-903ac8165eeb-signing-cabundle\") pod \"service-ca-7b994bc59-n8tmk\" (UID: \"c876f183-f50d-41bb-8d34-903ac8165eeb\") " pod="openshift-service-ca/service-ca-7b994bc59-n8tmk" Jun 15 09:19:18.301171 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:18.301131 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrvw6\" (UniqueName: \"kubernetes.io/projected/c876f183-f50d-41bb-8d34-903ac8165eeb-kube-api-access-mrvw6\") pod \"service-ca-7b994bc59-n8tmk\" (UID: \"c876f183-f50d-41bb-8d34-903ac8165eeb\") " pod="openshift-service-ca/service-ca-7b994bc59-n8tmk" Jun 15 09:19:18.401905 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:18.401872 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c876f183-f50d-41bb-8d34-903ac8165eeb-signing-key\") pod \"service-ca-7b994bc59-n8tmk\" (UID: \"c876f183-f50d-41bb-8d34-903ac8165eeb\") " pod="openshift-service-ca/service-ca-7b994bc59-n8tmk" Jun 15 09:19:18.401905 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:18.401904 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c876f183-f50d-41bb-8d34-903ac8165eeb-signing-cabundle\") pod \"service-ca-7b994bc59-n8tmk\" (UID: \"c876f183-f50d-41bb-8d34-903ac8165eeb\") " pod="openshift-service-ca/service-ca-7b994bc59-n8tmk" Jun 15 09:19:18.402068 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:18.401935 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrvw6\" (UniqueName: \"kubernetes.io/projected/c876f183-f50d-41bb-8d34-903ac8165eeb-kube-api-access-mrvw6\") pod \"service-ca-7b994bc59-n8tmk\" (UID: \"c876f183-f50d-41bb-8d34-903ac8165eeb\") " pod="openshift-service-ca/service-ca-7b994bc59-n8tmk" Jun 15 09:19:18.402545 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:18.402514 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c876f183-f50d-41bb-8d34-903ac8165eeb-signing-cabundle\") pod \"service-ca-7b994bc59-n8tmk\" (UID: \"c876f183-f50d-41bb-8d34-903ac8165eeb\") " pod="openshift-service-ca/service-ca-7b994bc59-n8tmk" Jun 15 09:19:18.404150 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:18.404125 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c876f183-f50d-41bb-8d34-903ac8165eeb-signing-key\") pod \"service-ca-7b994bc59-n8tmk\" (UID: \"c876f183-f50d-41bb-8d34-903ac8165eeb\") " pod="openshift-service-ca/service-ca-7b994bc59-n8tmk" Jun 15 09:19:18.409275 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:18.409248 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrvw6\" (UniqueName: \"kubernetes.io/projected/c876f183-f50d-41bb-8d34-903ac8165eeb-kube-api-access-mrvw6\") pod \"service-ca-7b994bc59-n8tmk\" (UID: \"c876f183-f50d-41bb-8d34-903ac8165eeb\") " pod="openshift-service-ca/service-ca-7b994bc59-n8tmk" Jun 15 09:19:18.526319 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:18.526294 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7b994bc59-n8tmk" Jun 15 09:19:18.660118 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:18.660090 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7b994bc59-n8tmk"] Jun 15 09:19:18.663989 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:19:18.663963 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc876f183_f50d_41bb_8d34_903ac8165eeb.slice/crio-2e51b127fc06144d885ad2ffe3c14de8021a6e363332a98d5f3aeeed5f3cdda0 WatchSource:0}: Error finding container 2e51b127fc06144d885ad2ffe3c14de8021a6e363332a98d5f3aeeed5f3cdda0: Status 404 returned error can't find the container with id 2e51b127fc06144d885ad2ffe3c14de8021a6e363332a98d5f3aeeed5f3cdda0 Jun 15 09:19:18.667049 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:18.667032 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jlhbm_eac64cde-28f4-49be-8543-42e3b4ed898f/dns-node-resolver/0.log" Jun 15 09:19:19.124981 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:19.124934 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:19.144957 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:19.144928 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-kdt7s" event={"ID":"4bb39dda-ff30-41eb-a023-8866e8df5889","Type":"ContainerStarted","Data":"af428beb799082d7ad892c30192066b72e34634fac8986dad86e46c0fdc4a8f8"} Jun 15 09:19:19.145118 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:19.144984 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-kdt7s" event={"ID":"4bb39dda-ff30-41eb-a023-8866e8df5889","Type":"ContainerStarted","Data":"da3900d5f70e9bb3daf20c62f2721e979c3bdabce94f4c758aee6616c07509c3"} Jun 15 09:19:19.146499 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:19.146430 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7b994bc59-n8tmk" event={"ID":"c876f183-f50d-41bb-8d34-903ac8165eeb","Type":"ContainerStarted","Data":"2e51b127fc06144d885ad2ffe3c14de8021a6e363332a98d5f3aeeed5f3cdda0"} Jun 15 09:19:19.162580 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:19.162523 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d4bcd4867-kdt7s" podStartSLOduration=2.072703172 podStartE2EDuration="4.162505056s" totalImagesPullingTime="2.089801884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:19:19.161570625 +0000 UTC m=+43.724992930" watchObservedRunningTime="2026-06-15 09:19:19.162505056 +0000 UTC m=+43.725927362" Jun 15 09:19:19.187361 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:19.187328 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:19.667627 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:19.667601 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lfzxm_a11b7428-db5b-4672-97f0-4cfaf32de873/node-ca/0.log" Jun 15 09:19:19.811176 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:19.811143 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79f2f4a8-307d-48e2-9c31-83bcd1636183-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kz4mb\" (UID: \"79f2f4a8-307d-48e2-9c31-83bcd1636183\") " pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:19.811353 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:19.811233 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79f2f4a8-307d-48e2-9c31-83bcd1636183-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kz4mb\" (UID: \"79f2f4a8-307d-48e2-9c31-83bcd1636183\") " pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:19.811353 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:19.811323 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 09:19:19.811456 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:19.811373 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f2f4a8-307d-48e2-9c31-83bcd1636183-insights-runtime-extractor-tls podName:79f2f4a8-307d-48e2-9c31-83bcd1636183 nodeName:}" failed. No retries permitted until 2026-06-15 09:19:23.811359988 +0000 UTC m=+48.374782271 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79f2f4a8-307d-48e2-9c31-83bcd1636183-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kz4mb" (UID: "79f2f4a8-307d-48e2-9c31-83bcd1636183") : secret "insights-runtime-extractor-tls" not found Jun 15 09:19:19.811688 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:19.811665 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79f2f4a8-307d-48e2-9c31-83bcd1636183-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kz4mb\" (UID: \"79f2f4a8-307d-48e2-9c31-83bcd1636183\") " pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:20.068070 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:20.068047 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gwnsl_22891e8c-12f1-48c2-910f-5015573cdad8/serve-healthcheck-canary/0.log" Jun 15 09:19:20.311671 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:20.311634 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:20.412290 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:20.412205 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:20.468533 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:20.468477 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d4bcd4867-kdt7s_4bb39dda-ff30-41eb-a023-8866e8df5889/migrator/0.log" Jun 15 09:19:20.667387 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:20.667304 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d4bcd4867-kdt7s_4bb39dda-ff30-41eb-a023-8866e8df5889/graceful-termination/0.log" Jun 15 09:19:21.153166 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:21.153135 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7b994bc59-n8tmk" event={"ID":"c876f183-f50d-41bb-8d34-903ac8165eeb","Type":"ContainerStarted","Data":"28bbe6fabed70302bf39b8a0b08927e88ddd056faa505548c3d1103d11eecd53"} Jun 15 09:19:21.167421 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:21.167372 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7b994bc59-n8tmk" podStartSLOduration=1.022034876 podStartE2EDuration="3.167354846s" totalImagesPullingTime="2.14531997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:19:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:19:21.166401097 +0000 UTC m=+45.729823457" watchObservedRunningTime="2026-06-15 09:19:21.167354846 +0000 UTC m=+45.730777151" Jun 15 09:19:21.978482 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:21.978432 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:22.080276 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:22.080235 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:22.214677 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:22.214647 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:22.285539 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:22.285447 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:22.392236 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:22.392204 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:22.558656 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:22.558571 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:22.668289 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:22.668250 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:23.670179 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:23.670144 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:23.790973 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:23.790943 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:23.844519 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:23.844490 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79f2f4a8-307d-48e2-9c31-83bcd1636183-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kz4mb\" (UID: \"79f2f4a8-307d-48e2-9c31-83bcd1636183\") " pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:23.844680 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:23.844592 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 09:19:23.844680 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:19:23.844640 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f2f4a8-307d-48e2-9c31-83bcd1636183-insights-runtime-extractor-tls podName:79f2f4a8-307d-48e2-9c31-83bcd1636183 nodeName:}" failed. No retries permitted until 2026-06-15 09:19:31.84462778 +0000 UTC m=+56.408050063 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79f2f4a8-307d-48e2-9c31-83bcd1636183-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kz4mb" (UID: "79f2f4a8-307d-48e2-9c31-83bcd1636183") : secret "insights-runtime-extractor-tls" not found Jun 15 09:19:24.081731 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:24.081702 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:24.265315 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:24.265278 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:24.467303 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:24.467227 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:24.665054 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:24.665022 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:25.558047 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:25.558016 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:25.667335 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:25.667307 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:25.768016 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:25.767981 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:25.870974 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:25.870940 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:26.060515 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:26.060487 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-tls\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:26.060773 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:26.060527 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/695a033a-993a-4bd2-9f73-3f979df52254-metrics-tls\") pod \"dns-default-rr6xd\" (UID: \"695a033a-993a-4bd2-9f73-3f979df52254\") " pod="openshift-dns/dns-default-rr6xd" Jun 15 09:19:26.062851 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:26.062808 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/695a033a-993a-4bd2-9f73-3f979df52254-metrics-tls\") pod \"dns-default-rr6xd\" (UID: \"695a033a-993a-4bd2-9f73-3f979df52254\") " pod="openshift-dns/dns-default-rr6xd" Jun 15 09:19:26.062965 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:26.062872 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-tls\") pod \"image-registry-66c98d86-pkr99\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:26.065828 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:26.065805 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:26.128758 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:26.128703 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rr6xd" Jun 15 09:19:26.136855 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:26.136821 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:26.269517 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:26.269468 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:26.297060 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:26.297030 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rr6xd"] Jun 15 09:19:26.301141 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:19:26.301111 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod695a033a_993a_4bd2_9f73_3f979df52254.slice/crio-315f842ccd2630afcd406b971c826f5b42078cf031917ba8244e67b7a4a6e8bd WatchSource:0}: Error finding container 315f842ccd2630afcd406b971c826f5b42078cf031917ba8244e67b7a4a6e8bd: Status 404 returned error can't find the container with id 315f842ccd2630afcd406b971c826f5b42078cf031917ba8244e67b7a4a6e8bd Jun 15 09:19:26.312166 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:26.312143 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66c98d86-pkr99"] Jun 15 09:19:26.314743 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:19:26.314722 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0690cee7_ba7d_4990_9e23_4d86f39cdd5c.slice/crio-41cbaa18ae59801793c78732383bcfe7935b89d6b7f1df0f256b265d2022defe WatchSource:0}: Error finding container 41cbaa18ae59801793c78732383bcfe7935b89d6b7f1df0f256b265d2022defe: Status 404 returned error can't find the container with id 41cbaa18ae59801793c78732383bcfe7935b89d6b7f1df0f256b265d2022defe Jun 15 09:19:26.466052 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:26.466013 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:26.514195 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:26.514162 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:26.664071 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:26.663990 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:27.172715 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:27.172680 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66c98d86-pkr99" event={"ID":"0690cee7-ba7d-4990-9e23-4d86f39cdd5c","Type":"ContainerStarted","Data":"33836c7cd7fa6c32ed8dcec04bb88cea8c893157d882493da0a069857a6b059a"} Jun 15 09:19:27.172932 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:27.172737 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66c98d86-pkr99" event={"ID":"0690cee7-ba7d-4990-9e23-4d86f39cdd5c","Type":"ContainerStarted","Data":"41cbaa18ae59801793c78732383bcfe7935b89d6b7f1df0f256b265d2022defe"} Jun 15 09:19:27.172932 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:27.172788 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:27.174077 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:27.174026 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rr6xd" event={"ID":"695a033a-993a-4bd2-9f73-3f979df52254","Type":"ContainerStarted","Data":"315f842ccd2630afcd406b971c826f5b42078cf031917ba8244e67b7a4a6e8bd"} Jun 15 09:19:27.191364 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:27.191311 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-66c98d86-pkr99" podStartSLOduration=48.191295137 podStartE2EDuration="48.191295137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:18:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:19:27.190414226 +0000 UTC m=+51.753836532" watchObservedRunningTime="2026-06-15 09:19:27.191295137 +0000 UTC m=+51.754717442" Jun 15 09:19:27.663264 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:27.663223 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:27.764462 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:27.764436 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:27.865796 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:27.865742 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:28.064772 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:28.064693 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:28.262668 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:28.262632 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:28.317448 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:28.317400 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:28.470106 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:28.470076 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:28.862283 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:28.862231 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:29.186014 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:29.185930 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rr6xd" event={"ID":"695a033a-993a-4bd2-9f73-3f979df52254","Type":"ContainerStarted","Data":"ca233d358ec5a777e174a1c42a3d3b510cf3d6872e730a2d60610335ccb0df78"} Jun 15 09:19:29.186014 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:29.185988 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rr6xd" event={"ID":"695a033a-993a-4bd2-9f73-3f979df52254","Type":"ContainerStarted","Data":"1380811a8b7d3db06ccaf42706835794ae77164e164ef0379865b3e431f58d12"} Jun 15 09:19:29.186197 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:29.186031 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rr6xd" Jun 15 09:19:29.202978 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:29.202933 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rr6xd" podStartSLOduration=16.884771063 podStartE2EDuration="19.202917958s" totalImagesPullingTime="2.318146895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:19:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:19:29.201771001 +0000 UTC m=+53.765193308" watchObservedRunningTime="2026-06-15 09:19:29.202917958 +0000 UTC m=+53.766340262" Jun 15 09:19:29.466080 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:29.466007 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:29.659604 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:29.659569 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:30.265535 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:30.265506 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:30.664323 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:30.664293 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:31.065782 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:31.065694 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:31.264218 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:31.264186 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:31.870364 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:31.870328 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:31.904122 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:31.904097 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79f2f4a8-307d-48e2-9c31-83bcd1636183-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kz4mb\" (UID: \"79f2f4a8-307d-48e2-9c31-83bcd1636183\") " pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:31.906470 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:31.906453 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79f2f4a8-307d-48e2-9c31-83bcd1636183-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kz4mb\" (UID: \"79f2f4a8-307d-48e2-9c31-83bcd1636183\") " pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:32.011348 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:32.011321 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kz4mb" Jun 15 09:19:32.132852 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:32.132774 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kz4mb"] Jun 15 09:19:32.135585 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:19:32.135553 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79f2f4a8_307d_48e2_9c31_83bcd1636183.slice/crio-5a003c5e30615d19cb9e0a01feba5c97a42a133e498319df40556436479c90a5 WatchSource:0}: Error finding container 5a003c5e30615d19cb9e0a01feba5c97a42a133e498319df40556436479c90a5: Status 404 returned error can't find the container with id 5a003c5e30615d19cb9e0a01feba5c97a42a133e498319df40556436479c90a5 Jun 15 09:19:32.195110 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:32.195083 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kz4mb" event={"ID":"79f2f4a8-307d-48e2-9c31-83bcd1636183","Type":"ContainerStarted","Data":"5a003c5e30615d19cb9e0a01feba5c97a42a133e498319df40556436479c90a5"} Jun 15 09:19:32.269217 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:32.269187 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:32.460612 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:32.460531 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:32.664073 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:32.664030 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:35.319918 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:35.319878 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:35.370901 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:35.370860 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:36.106092 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:36.106072 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pxdxj" Jun 15 09:19:36.210107 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:36.210080 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kz4mb" event={"ID":"79f2f4a8-307d-48e2-9c31-83bcd1636183","Type":"ContainerStarted","Data":"a37f8eb5a08f3cd6266c2d9835f23e04f8955f6a73498816110d882456f9784e"} Jun 15 09:19:36.210203 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:36.210130 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kz4mb" event={"ID":"79f2f4a8-307d-48e2-9c31-83bcd1636183","Type":"ContainerStarted","Data":"bead1d3c18661a7371dca2ea5f4766ab64d17d2437b8036c9a0a52f0e8e48f56"} Jun 15 09:19:38.217807 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:38.217774 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kz4mb" event={"ID":"79f2f4a8-307d-48e2-9c31-83bcd1636183","Type":"ContainerStarted","Data":"8ae9792698d696a6a173ac7fad888289ef0384526a85f6cace1af7f3a22c7c8e"} Jun 15 09:19:38.237011 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:38.236961 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kz4mb" podStartSLOduration=16.471724785 podStartE2EDuration="22.236939763s" totalImagesPullingTime="5.765214978s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 09:19:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:19:38.23576776 +0000 UTC m=+62.799190066" watchObservedRunningTime="2026-06-15 09:19:38.236939763 +0000 UTC m=+62.800362069" Jun 15 09:19:38.979002 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:38.978971 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b497fb477-nkvdg"] Jun 15 09:19:38.980703 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:38.980679 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-597bbbd8cd-dhpnx"] Jun 15 09:19:38.980868 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:38.980814 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b497fb477-nkvdg" Jun 15 09:19:38.982568 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:38.982550 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-597bbbd8cd-dhpnx" Jun 15 09:19:38.983509 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:38.983482 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 15 09:19:38.983509 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:38.983506 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 15 09:19:38.984347 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:38.984321 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 15 09:19:38.984445 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:38.984336 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 15 09:19:38.985926 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:38.985904 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jun 15 09:19:38.986177 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:38.986155 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-4pn9f\"" Jun 15 09:19:38.994181 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:38.994160 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-597bbbd8cd-dhpnx"] Jun 15 09:19:38.994917 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:38.994897 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b497fb477-nkvdg"] Jun 15 09:19:38.999111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:38.999089 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66c98d86-pkr99"] Jun 15 09:19:39.059943 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.059916 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef4126e9-ae75-4bae-89a4-d117cc1e88bd-tmp\") pod \"klusterlet-addon-workmgr-7b497fb477-nkvdg\" (UID: \"ef4126e9-ae75-4bae-89a4-d117cc1e88bd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b497fb477-nkvdg" Jun 15 09:19:39.060058 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.059947 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgh8d\" (UniqueName: \"kubernetes.io/projected/ef4126e9-ae75-4bae-89a4-d117cc1e88bd-kube-api-access-kgh8d\") pod \"klusterlet-addon-workmgr-7b497fb477-nkvdg\" (UID: \"ef4126e9-ae75-4bae-89a4-d117cc1e88bd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b497fb477-nkvdg" Jun 15 09:19:39.060058 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.059993 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbf6z\" (UniqueName: \"kubernetes.io/projected/f26086b3-7bd0-4742-9dd9-2f9890d934b5-kube-api-access-wbf6z\") pod \"managed-serviceaccount-addon-agent-597bbbd8cd-dhpnx\" (UID: \"f26086b3-7bd0-4742-9dd9-2f9890d934b5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-597bbbd8cd-dhpnx" Jun 15 09:19:39.060129 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.060056 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f26086b3-7bd0-4742-9dd9-2f9890d934b5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-597bbbd8cd-dhpnx\" (UID: \"f26086b3-7bd0-4742-9dd9-2f9890d934b5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-597bbbd8cd-dhpnx" Jun 15 09:19:39.060129 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.060085 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ef4126e9-ae75-4bae-89a4-d117cc1e88bd-klusterlet-config\") pod \"klusterlet-addon-workmgr-7b497fb477-nkvdg\" (UID: \"ef4126e9-ae75-4bae-89a4-d117cc1e88bd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b497fb477-nkvdg" Jun 15 09:19:39.096509 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.096482 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:39.161245 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.161219 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgh8d\" (UniqueName: \"kubernetes.io/projected/ef4126e9-ae75-4bae-89a4-d117cc1e88bd-kube-api-access-kgh8d\") pod \"klusterlet-addon-workmgr-7b497fb477-nkvdg\" (UID: \"ef4126e9-ae75-4bae-89a4-d117cc1e88bd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b497fb477-nkvdg" Jun 15 09:19:39.161363 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.161266 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbf6z\" (UniqueName: \"kubernetes.io/projected/f26086b3-7bd0-4742-9dd9-2f9890d934b5-kube-api-access-wbf6z\") pod \"managed-serviceaccount-addon-agent-597bbbd8cd-dhpnx\" (UID: \"f26086b3-7bd0-4742-9dd9-2f9890d934b5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-597bbbd8cd-dhpnx" Jun 15 09:19:39.161363 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.161330 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f26086b3-7bd0-4742-9dd9-2f9890d934b5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-597bbbd8cd-dhpnx\" (UID: \"f26086b3-7bd0-4742-9dd9-2f9890d934b5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-597bbbd8cd-dhpnx" Jun 15 09:19:39.161363 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.161358 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ef4126e9-ae75-4bae-89a4-d117cc1e88bd-klusterlet-config\") pod \"klusterlet-addon-workmgr-7b497fb477-nkvdg\" (UID: \"ef4126e9-ae75-4bae-89a4-d117cc1e88bd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b497fb477-nkvdg" Jun 15 09:19:39.161509 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.161405 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef4126e9-ae75-4bae-89a4-d117cc1e88bd-tmp\") pod \"klusterlet-addon-workmgr-7b497fb477-nkvdg\" (UID: \"ef4126e9-ae75-4bae-89a4-d117cc1e88bd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b497fb477-nkvdg" Jun 15 09:19:39.161722 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.161701 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef4126e9-ae75-4bae-89a4-d117cc1e88bd-tmp\") pod \"klusterlet-addon-workmgr-7b497fb477-nkvdg\" (UID: \"ef4126e9-ae75-4bae-89a4-d117cc1e88bd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b497fb477-nkvdg" Jun 15 09:19:39.163762 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.163733 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f26086b3-7bd0-4742-9dd9-2f9890d934b5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-597bbbd8cd-dhpnx\" (UID: \"f26086b3-7bd0-4742-9dd9-2f9890d934b5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-597bbbd8cd-dhpnx" Jun 15 09:19:39.163872 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.163795 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ef4126e9-ae75-4bae-89a4-d117cc1e88bd-klusterlet-config\") pod \"klusterlet-addon-workmgr-7b497fb477-nkvdg\" (UID: \"ef4126e9-ae75-4bae-89a4-d117cc1e88bd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b497fb477-nkvdg" Jun 15 09:19:39.173801 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.173782 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbf6z\" (UniqueName: \"kubernetes.io/projected/f26086b3-7bd0-4742-9dd9-2f9890d934b5-kube-api-access-wbf6z\") pod \"managed-serviceaccount-addon-agent-597bbbd8cd-dhpnx\" (UID: \"f26086b3-7bd0-4742-9dd9-2f9890d934b5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-597bbbd8cd-dhpnx" Jun 15 09:19:39.173801 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.173796 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgh8d\" (UniqueName: \"kubernetes.io/projected/ef4126e9-ae75-4bae-89a4-d117cc1e88bd-kube-api-access-kgh8d\") pod \"klusterlet-addon-workmgr-7b497fb477-nkvdg\" (UID: \"ef4126e9-ae75-4bae-89a4-d117cc1e88bd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b497fb477-nkvdg" Jun 15 09:19:39.180213 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.180190 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:39.190659 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.190643 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rr6xd" Jun 15 09:19:39.294982 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.294904 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b497fb477-nkvdg" Jun 15 09:19:39.308445 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.308424 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-597bbbd8cd-dhpnx" Jun 15 09:19:39.317009 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.316979 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:39.444849 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.444804 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b497fb477-nkvdg"] Jun 15 09:19:39.447990 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.447963 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:39.449227 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:19:39.449204 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef4126e9_ae75_4bae_89a4_d117cc1e88bd.slice/crio-80af8129307ac3519167d89e851f8f7e5499deee407dfddb8dc15462edf36b23 WatchSource:0}: Error finding container 80af8129307ac3519167d89e851f8f7e5499deee407dfddb8dc15462edf36b23: Status 404 returned error can't find the container with id 80af8129307ac3519167d89e851f8f7e5499deee407dfddb8dc15462edf36b23 Jun 15 09:19:39.463402 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:39.463382 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-597bbbd8cd-dhpnx"] Jun 15 09:19:39.466448 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:19:39.466430 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf26086b3_7bd0_4742_9dd9_2f9890d934b5.slice/crio-deec83dc330652d368f60ae8e27948c164b2f62e7a1d1d3f4796b720e20d8444 WatchSource:0}: Error finding container deec83dc330652d368f60ae8e27948c164b2f62e7a1d1d3f4796b720e20d8444: Status 404 returned error can't find the container with id deec83dc330652d368f60ae8e27948c164b2f62e7a1d1d3f4796b720e20d8444 Jun 15 09:19:40.227905 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:40.227768 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-597bbbd8cd-dhpnx" event={"ID":"f26086b3-7bd0-4742-9dd9-2f9890d934b5","Type":"ContainerStarted","Data":"deec83dc330652d368f60ae8e27948c164b2f62e7a1d1d3f4796b720e20d8444"} Jun 15 09:19:40.229753 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:40.229696 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b497fb477-nkvdg" event={"ID":"ef4126e9-ae75-4bae-89a4-d117cc1e88bd","Type":"ContainerStarted","Data":"80af8129307ac3519167d89e851f8f7e5499deee407dfddb8dc15462edf36b23"} Jun 15 09:19:41.686057 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:41.686015 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs\") pod \"network-metrics-daemon-7trxt\" (UID: \"67c71a8c-71f7-4494-a8cd-fd7b2af8ab51\") " pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:19:41.688480 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:41.688457 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 15 09:19:41.699568 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:41.699540 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67c71a8c-71f7-4494-a8cd-fd7b2af8ab51-metrics-certs\") pod \"network-metrics-daemon-7trxt\" (UID: \"67c71a8c-71f7-4494-a8cd-fd7b2af8ab51\") " pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:19:41.786931 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:41.786885 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret\") pod \"global-pull-secret-syncer-kddf5\" (UID: \"56a0c215-5d6d-4cb2-b9aa-505333bc0cea\") " pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:19:41.789650 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:41.789614 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 15 09:19:41.800887 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:41.800859 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/56a0c215-5d6d-4cb2-b9aa-505333bc0cea-original-pull-secret\") pod \"global-pull-secret-syncer-kddf5\" (UID: \"56a0c215-5d6d-4cb2-b9aa-505333bc0cea\") " pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:19:41.842068 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:41.842040 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kddf5" Jun 15 09:19:41.850586 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:41.850564 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vhk77\"" Jun 15 09:19:41.858613 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:41.858591 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7trxt" Jun 15 09:19:41.887637 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:41.887605 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2vs8\" (UniqueName: \"kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8\") pod \"network-check-target-cdmf7\" (UID: \"4e83c2a6-9354-4e58-85fc-9f351b1991d7\") " pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:19:41.890222 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:41.890195 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 15 09:19:41.900244 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:41.900219 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 15 09:19:41.912065 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:41.912035 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2vs8\" (UniqueName: \"kubernetes.io/projected/4e83c2a6-9354-4e58-85fc-9f351b1991d7-kube-api-access-x2vs8\") pod \"network-check-target-cdmf7\" (UID: \"4e83c2a6-9354-4e58-85fc-9f351b1991d7\") " pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:19:42.156489 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:42.156456 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hsb9v\"" Jun 15 09:19:42.164506 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:42.164483 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:19:44.144985 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:44.144947 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:44.265877 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:44.265496 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:44.324350 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:44.324319 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:44.369124 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:44.369085 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kddf5"] Jun 15 09:19:44.380031 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:44.379928 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7trxt"] Jun 15 09:19:44.404882 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:44.404781 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cdmf7"] Jun 15 09:19:44.437893 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:44.437854 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:44.593416 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:19:44.593370 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56a0c215_5d6d_4cb2_b9aa_505333bc0cea.slice/crio-38ac4d3e7c2fb11861dce976f2cc1a748b2ac901f76513d52b2103b5072f56c2 WatchSource:0}: Error finding container 38ac4d3e7c2fb11861dce976f2cc1a748b2ac901f76513d52b2103b5072f56c2: Status 404 returned error can't find the container with id 38ac4d3e7c2fb11861dce976f2cc1a748b2ac901f76513d52b2103b5072f56c2 Jun 15 09:19:44.594272 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:19:44.594229 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67c71a8c_71f7_4494_a8cd_fd7b2af8ab51.slice/crio-8f7dcb50ae1a33de5feb95fc109da63b7b11058f23e1e056c435e13c40b89387 WatchSource:0}: Error finding container 8f7dcb50ae1a33de5feb95fc109da63b7b11058f23e1e056c435e13c40b89387: Status 404 returned error can't find the container with id 8f7dcb50ae1a33de5feb95fc109da63b7b11058f23e1e056c435e13c40b89387 Jun 15 09:19:44.595238 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:19:44.595114 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e83c2a6_9354_4e58_85fc_9f351b1991d7.slice/crio-a307163217bf5a07f9f38636d07b0c95129f820dffd8f5b07a2ef28d389e09d7 WatchSource:0}: Error finding container a307163217bf5a07f9f38636d07b0c95129f820dffd8f5b07a2ef28d389e09d7: Status 404 returned error can't find the container with id a307163217bf5a07f9f38636d07b0c95129f820dffd8f5b07a2ef28d389e09d7 Jun 15 09:19:44.650307 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:44.650284 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:44.728231 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:44.728200 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:44.934362 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:44.934266 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:45.092876 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:45.092820 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:45.175945 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:45.175873 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:45.237082 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:45.236972 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:45.263596 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:45.263564 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-597bbbd8cd-dhpnx" event={"ID":"f26086b3-7bd0-4742-9dd9-2f9890d934b5","Type":"ContainerStarted","Data":"428ec5355e2a92d38c5fd5cf3e236819b8adac7e28d42e53c3ec8d3bfadf9a03"} Jun 15 09:19:45.267824 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:45.267738 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cdmf7" event={"ID":"4e83c2a6-9354-4e58-85fc-9f351b1991d7","Type":"ContainerStarted","Data":"a307163217bf5a07f9f38636d07b0c95129f820dffd8f5b07a2ef28d389e09d7"} Jun 15 09:19:45.269550 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:45.269517 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7trxt" event={"ID":"67c71a8c-71f7-4494-a8cd-fd7b2af8ab51","Type":"ContainerStarted","Data":"8f7dcb50ae1a33de5feb95fc109da63b7b11058f23e1e056c435e13c40b89387"} Jun 15 09:19:45.272652 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:45.272623 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b497fb477-nkvdg" event={"ID":"ef4126e9-ae75-4bae-89a4-d117cc1e88bd","Type":"ContainerStarted","Data":"a6f9d67ea03dcb7db165670d7907b1ff554c94a7374f691c73e06dfc17904963"} Jun 15 09:19:45.273247 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:45.273228 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b497fb477-nkvdg" Jun 15 09:19:45.275556 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:45.275505 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b497fb477-nkvdg" Jun 15 09:19:45.275685 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:45.275646 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kddf5" event={"ID":"56a0c215-5d6d-4cb2-b9aa-505333bc0cea","Type":"ContainerStarted","Data":"38ac4d3e7c2fb11861dce976f2cc1a748b2ac901f76513d52b2103b5072f56c2"} Jun 15 09:19:45.296850 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:45.296793 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-597bbbd8cd-dhpnx" podStartSLOduration=2.576994133 podStartE2EDuration="7.296776843s" totalImagesPullingTime="4.71978271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:19:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:19:45.279598615 +0000 UTC m=+69.843020921" watchObservedRunningTime="2026-06-15 09:19:45.296776843 +0000 UTC m=+69.860199152" Jun 15 09:19:45.370192 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:45.370143 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:45.717170 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:45.717127 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:46.284051 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:46.283388 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7trxt" event={"ID":"67c71a8c-71f7-4494-a8cd-fd7b2af8ab51","Type":"ContainerStarted","Data":"29a3c8aedf1d997a6ece583b8b11daf8f8411a8da11a268c16f8ad338da670b7"} Jun 15 09:19:46.320467 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:46.320205 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:46.521863 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:46.521748 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:46.770338 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:46.770305 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:47.218037 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:47.218002 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:47.267978 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:47.267876 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:47.290373 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:47.290277 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7trxt" event={"ID":"67c71a8c-71f7-4494-a8cd-fd7b2af8ab51","Type":"ContainerStarted","Data":"f6eeb3239ff56cc68931cc9802b6b004ffca2327f4c90e1251c599e14846bef5"} Jun 15 09:19:47.310211 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:47.310161 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7trxt" podStartSLOduration=69.821923636 podStartE2EDuration="1m11.310144685s" totalImagesPullingTime="1.488221049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:18:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:19:47.308539067 +0000 UTC m=+71.871961388" watchObservedRunningTime="2026-06-15 09:19:47.310144685 +0000 UTC m=+71.873566990" Jun 15 09:19:47.310980 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:47.310945 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b497fb477-nkvdg" podStartSLOduration=4.122200337 podStartE2EDuration="9.310931951s" totalImagesPullingTime="5.188731614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:19:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:19:45.297767302 +0000 UTC m=+69.861189632" watchObservedRunningTime="2026-06-15 09:19:47.310931951 +0000 UTC m=+71.874354259" Jun 15 09:19:47.338268 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:47.338211 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:47.439894 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:47.439856 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:47.539782 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:47.539702 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:47.592691 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:47.592649 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:47.787382 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:47.787346 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:48.318434 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:48.318399 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:48.370696 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:48.370654 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:49.006634 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:49.006602 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:19:49.972709 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:49.972667 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:50.303025 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:50.302929 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cdmf7" event={"ID":"4e83c2a6-9354-4e58-85fc-9f351b1991d7","Type":"ContainerStarted","Data":"ebcd349f2692d0ec08b52779e2a663ca860b94056c5e4acda7f7144831c5306b"} Jun 15 09:19:50.303188 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:50.303036 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:19:50.304351 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:50.304326 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kddf5" event={"ID":"56a0c215-5d6d-4cb2-b9aa-505333bc0cea","Type":"ContainerStarted","Data":"ba18caddbacef3e0278c17388099823b40d73b1b4aafd3e37b4fd3d977e24599"} Jun 15 09:19:50.316726 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:50.316674 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-cdmf7" podStartSLOduration=69.478499627 podStartE2EDuration="1m14.316657646s" totalImagesPullingTime="4.838158019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:18:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:19:50.316339471 +0000 UTC m=+74.879761780" watchObservedRunningTime="2026-06-15 09:19:50.316657646 +0000 UTC m=+74.880079950" Jun 15 09:19:50.334489 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:50.334439 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-kddf5" podStartSLOduration=69.49412162 podStartE2EDuration="1m14.334419945s" totalImagesPullingTime="4.840298325s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 09:18:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:19:50.333556535 +0000 UTC m=+74.896978878" watchObservedRunningTime="2026-06-15 09:19:50.334419945 +0000 UTC m=+74.897842250" Jun 15 09:19:51.477786 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.477748 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:51.613165 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.613132 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:51.724940 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.724906 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:51.801219 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.801112 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-sv9zr"] Jun 15 09:19:51.805582 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.805561 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.809699 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.809674 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 15 09:19:51.809857 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.809755 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-45jk6\"" Jun 15 09:19:51.810880 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.810861 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 15 09:19:51.811088 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.810867 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 15 09:19:51.811228 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.810906 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 15 09:19:51.811350 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.810907 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 15 09:19:51.811429 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.810916 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 15 09:19:51.847167 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.847133 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:51.868103 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.868068 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns5z4\" (UniqueName: \"kubernetes.io/projected/9f44c5b9-3847-4f50-bc6f-809b7569c288-kube-api-access-ns5z4\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.868263 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.868146 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9f44c5b9-3847-4f50-bc6f-809b7569c288-root\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.868263 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.868196 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9f44c5b9-3847-4f50-bc6f-809b7569c288-node-exporter-accelerators-collector-config\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.868263 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.868222 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f44c5b9-3847-4f50-bc6f-809b7569c288-metrics-client-ca\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.868383 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.868315 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9f44c5b9-3847-4f50-bc6f-809b7569c288-node-exporter-wtmp\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.868383 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.868345 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9f44c5b9-3847-4f50-bc6f-809b7569c288-node-exporter-tls\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.868457 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.868392 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f44c5b9-3847-4f50-bc6f-809b7569c288-sys\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.868457 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.868419 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9f44c5b9-3847-4f50-bc6f-809b7569c288-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.868553 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.868455 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9f44c5b9-3847-4f50-bc6f-809b7569c288-node-exporter-textfile\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.940902 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.940865 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:51.969102 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.969067 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9f44c5b9-3847-4f50-bc6f-809b7569c288-node-exporter-wtmp\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.969267 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.969122 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9f44c5b9-3847-4f50-bc6f-809b7569c288-node-exporter-tls\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.969267 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.969174 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f44c5b9-3847-4f50-bc6f-809b7569c288-sys\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.969267 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.969202 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9f44c5b9-3847-4f50-bc6f-809b7569c288-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.969267 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.969231 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9f44c5b9-3847-4f50-bc6f-809b7569c288-node-exporter-textfile\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.969267 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.969247 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9f44c5b9-3847-4f50-bc6f-809b7569c288-node-exporter-wtmp\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.969542 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.969254 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns5z4\" (UniqueName: \"kubernetes.io/projected/9f44c5b9-3847-4f50-bc6f-809b7569c288-kube-api-access-ns5z4\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.969542 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.969313 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9f44c5b9-3847-4f50-bc6f-809b7569c288-root\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.969542 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.969348 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9f44c5b9-3847-4f50-bc6f-809b7569c288-node-exporter-accelerators-collector-config\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.969542 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.969374 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f44c5b9-3847-4f50-bc6f-809b7569c288-metrics-client-ca\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.969754 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.969572 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f44c5b9-3847-4f50-bc6f-809b7569c288-sys\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.969754 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.969664 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9f44c5b9-3847-4f50-bc6f-809b7569c288-root\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.970012 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.969985 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9f44c5b9-3847-4f50-bc6f-809b7569c288-node-exporter-textfile\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.970084 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.970043 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f44c5b9-3847-4f50-bc6f-809b7569c288-metrics-client-ca\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.970636 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.970611 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9f44c5b9-3847-4f50-bc6f-809b7569c288-node-exporter-accelerators-collector-config\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.972478 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.972453 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9f44c5b9-3847-4f50-bc6f-809b7569c288-node-exporter-tls\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.972584 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.972477 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9f44c5b9-3847-4f50-bc6f-809b7569c288-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:51.986790 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:51.986766 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns5z4\" (UniqueName: \"kubernetes.io/projected/9f44c5b9-3847-4f50-bc6f-809b7569c288-kube-api-access-ns5z4\") pod \"node-exporter-sv9zr\" (UID: \"9f44c5b9-3847-4f50-bc6f-809b7569c288\") " pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:52.038069 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:52.038035 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:52.116950 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:52.116915 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sv9zr" Jun 15 09:19:52.129194 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:19:52.129167 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f44c5b9_3847_4f50_bc6f_809b7569c288.slice/crio-b048b2562db94f9a4822a8ed673b674062c2073f21789488c9c56e018a3f953f WatchSource:0}: Error finding container b048b2562db94f9a4822a8ed673b674062c2073f21789488c9c56e018a3f953f: Status 404 returned error can't find the container with id b048b2562db94f9a4822a8ed673b674062c2073f21789488c9c56e018a3f953f Jun 15 09:19:52.143051 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:52.143014 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:52.237919 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:52.237880 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:52.311885 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:52.311824 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sv9zr" event={"ID":"9f44c5b9-3847-4f50-bc6f-809b7569c288","Type":"ContainerStarted","Data":"b048b2562db94f9a4822a8ed673b674062c2073f21789488c9c56e018a3f953f"} Jun 15 09:19:52.741467 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:52.741425 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:52.827530 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:52.827496 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:53.058700 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:53.058623 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:53.317523 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:53.317493 2562 generic.go:356] "Generic (PLEG): container finished" podID="9f44c5b9-3847-4f50-bc6f-809b7569c288" containerID="4a5abb53a4c22fb7c176aa079819bf8b4a1a098c8808418719d2efdb8e204224" exitCode=0 Jun 15 09:19:53.317674 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:53.317571 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sv9zr" event={"ID":"9f44c5b9-3847-4f50-bc6f-809b7569c288","Type":"ContainerDied","Data":"4a5abb53a4c22fb7c176aa079819bf8b4a1a098c8808418719d2efdb8e204224"} Jun 15 09:19:54.323120 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:54.323092 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sv9zr" event={"ID":"9f44c5b9-3847-4f50-bc6f-809b7569c288","Type":"ContainerStarted","Data":"38efc13383eccceb1298f0b99dc2cd62123a2e934c5c2fb359fedfad1120374e"} Jun 15 09:19:54.323476 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:54.323140 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sv9zr" event={"ID":"9f44c5b9-3847-4f50-bc6f-809b7569c288","Type":"ContainerStarted","Data":"3f3610ea35a7991fc16dd81d46b6e7e40042d6932712eca5ce7b2423aa0c8e99"} Jun 15 09:19:54.346426 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:54.343315 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-sv9zr" podStartSLOduration=2.345068146 podStartE2EDuration="3.343297942s" totalImagesPullingTime="934.80429ms" totalInitContainerRuntime="63.425506ms" isStatefulPod=true podCreationTimestamp="2026-06-15 09:19:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:19:54.341326731 +0000 UTC m=+78.904749056" watchObservedRunningTime="2026-06-15 09:19:54.343297942 +0000 UTC m=+78.906720247" Jun 15 09:19:54.678128 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:54.677981 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:54.779447 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:54.779416 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:54.838775 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:54.838741 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:55.062227 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:55.062148 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:56.276464 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:56.276427 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:56.339003 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:56.338969 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:56.489857 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:56.489804 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:56.550207 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:56.550126 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:56.617937 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:56.617900 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:57.942467 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:57.942433 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:57.944654 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:57.944631 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 15 09:19:57.949301 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:57.949286 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:57.956536 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:57.956521 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 15 09:19:57.956655 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:57.956633 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 15 09:19:57.956766 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:57.956709 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 15 09:19:57.956766 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:57.956714 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 15 09:19:57.957327 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:57.957307 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 15 09:19:57.958291 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:57.958269 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 15 09:19:57.958936 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:57.958868 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 15 09:19:57.959054 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:57.958948 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7odhctu18u789\"" Jun 15 09:19:57.959134 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:57.959102 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-f5dg2\"" Jun 15 09:19:57.959614 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:57.958500 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 15 09:19:57.959614 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:57.959446 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 15 09:19:57.960611 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:57.960453 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 15 09:19:57.960896 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:57.960830 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 15 09:19:57.961145 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:57.961129 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 15 09:19:57.961376 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:57.961357 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 15 09:19:57.962201 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:57.962183 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 15 09:19:57.963458 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:57.963444 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 15 09:19:57.975380 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:57.975357 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 15 09:19:58.113248 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.113216 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.113248 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.113251 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-config\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.113447 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.113275 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.113447 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.113357 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.113447 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.113398 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.113447 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.113419 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-web-config\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.113447 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.113439 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.113622 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.113485 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.113622 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.113503 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.113622 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.113528 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p5t9\" (UniqueName: \"kubernetes.io/projected/7aab4282-57ea-4f2b-a4f4-008802f60066-kube-api-access-8p5t9\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.113622 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.113556 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.113622 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.113594 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7aab4282-57ea-4f2b-a4f4-008802f60066-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.113622 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.113620 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.113908 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.113646 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.113908 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.113671 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.113908 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.113686 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.113908 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.113725 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.113908 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.113750 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.113908 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.113779 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7aab4282-57ea-4f2b-a4f4-008802f60066-config-out\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.113908 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.113863 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.214718 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.214623 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.214718 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.214670 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7aab4282-57ea-4f2b-a4f4-008802f60066-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.214718 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.214691 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.214718 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.214711 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.214718 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.214733 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.215123 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.214863 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.215123 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.214901 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.215123 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.214928 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.215123 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.214956 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7aab4282-57ea-4f2b-a4f4-008802f60066-config-out\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.215123 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.214987 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.215123 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.215022 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.215123 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.215062 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-config\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.215123 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.215097 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.215556 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.215154 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.215556 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.215184 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.215556 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.215206 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-web-config\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.215556 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.215232 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.215556 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.215257 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.215556 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.215282 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.215556 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.215310 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p5t9\" (UniqueName: \"kubernetes.io/projected/7aab4282-57ea-4f2b-a4f4-008802f60066-kube-api-access-8p5t9\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.216350 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.216325 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.216495 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.216414 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.216613 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.216587 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.216979 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.216954 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.217936 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.217915 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.219069 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.218853 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.219069 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.219026 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.219224 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.219170 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.219391 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.219369 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.219579 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.219556 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.220435 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.220410 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.220848 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.220775 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.221243 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.221204 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-web-config\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.221560 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.221539 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7aab4282-57ea-4f2b-a4f4-008802f60066-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.221635 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.221593 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.222229 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.222187 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.222325 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.222258 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.222399 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.222372 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7aab4282-57ea-4f2b-a4f4-008802f60066-config-out\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.222983 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.222961 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-config\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.223772 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.223753 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p5t9\" (UniqueName: \"kubernetes.io/projected/7aab4282-57ea-4f2b-a4f4-008802f60066-kube-api-access-8p5t9\") pod \"prometheus-k8s-0\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.260371 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.260351 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:19:58.396785 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.396749 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 15 09:19:58.399549 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:19:58.399527 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aab4282_57ea_4f2b_a4f4_008802f60066.slice/crio-cb476189d805e6172cd33b3338cb02042d98aca4a0048fa55ed7510d6fce6460 WatchSource:0}: Error finding container cb476189d805e6172cd33b3338cb02042d98aca4a0048fa55ed7510d6fce6460: Status 404 returned error can't find the container with id cb476189d805e6172cd33b3338cb02042d98aca4a0048fa55ed7510d6fce6460 Jun 15 09:19:58.735693 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.735659 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:58.786432 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:58.786408 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:59.129770 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:59.129718 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:59.178332 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:59.178295 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:19:59.341695 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:19:59.341650 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7aab4282-57ea-4f2b-a4f4-008802f60066","Type":"ContainerStarted","Data":"cb476189d805e6172cd33b3338cb02042d98aca4a0048fa55ed7510d6fce6460"} Jun 15 09:20:00.305369 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:00.305335 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:00.346700 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:00.346670 2562 generic.go:356] "Generic (PLEG): container finished" podID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerID="a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840" exitCode=0 Jun 15 09:20:00.346881 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:00.346760 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7aab4282-57ea-4f2b-a4f4-008802f60066","Type":"ContainerDied","Data":"a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840"} Jun 15 09:20:01.936108 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:01.936065 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:02.005556 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:02.005523 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:02.131469 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:02.131422 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:02.233440 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:02.233344 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:02.290593 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:02.290561 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:02.347853 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:02.347807 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:02.402590 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:02.402558 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:04.019784 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.019744 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-66c98d86-pkr99" podUID="0690cee7-ba7d-4990-9e23-4d86f39cdd5c" containerName="registry" containerID="cri-o://33836c7cd7fa6c32ed8dcec04bb88cea8c893157d882493da0a069857a6b059a" gracePeriod=30 Jun 15 09:20:04.262278 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.262255 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:20:04.363802 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.363762 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7aab4282-57ea-4f2b-a4f4-008802f60066","Type":"ContainerStarted","Data":"858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155"} Jun 15 09:20:04.364086 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.363909 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7aab4282-57ea-4f2b-a4f4-008802f60066","Type":"ContainerStarted","Data":"e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689"} Jun 15 09:20:04.365229 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.365199 2562 generic.go:356] "Generic (PLEG): container finished" podID="0690cee7-ba7d-4990-9e23-4d86f39cdd5c" containerID="33836c7cd7fa6c32ed8dcec04bb88cea8c893157d882493da0a069857a6b059a" exitCode=0 Jun 15 09:20:04.365341 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.365242 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66c98d86-pkr99" event={"ID":"0690cee7-ba7d-4990-9e23-4d86f39cdd5c","Type":"ContainerDied","Data":"33836c7cd7fa6c32ed8dcec04bb88cea8c893157d882493da0a069857a6b059a"} Jun 15 09:20:04.365341 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.365277 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66c98d86-pkr99" event={"ID":"0690cee7-ba7d-4990-9e23-4d86f39cdd5c","Type":"ContainerDied","Data":"41cbaa18ae59801793c78732383bcfe7935b89d6b7f1df0f256b265d2022defe"} Jun 15 09:20:04.365341 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.365284 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66c98d86-pkr99" Jun 15 09:20:04.365341 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.365304 2562 scope.go:117] "RemoveContainer" containerID="33836c7cd7fa6c32ed8dcec04bb88cea8c893157d882493da0a069857a6b059a" Jun 15 09:20:04.365553 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.365470 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-trusted-ca\") pod \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " Jun 15 09:20:04.365553 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.365506 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-installation-pull-secrets\") pod \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " Jun 15 09:20:04.365553 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.365532 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-tls\") pod \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " Jun 15 09:20:04.365710 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.365580 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-certificates\") pod \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " Jun 15 09:20:04.365710 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.365618 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-image-registry-private-configuration\") pod \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " Jun 15 09:20:04.365710 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.365644 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmxw7\" (UniqueName: \"kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-kube-api-access-hmxw7\") pod \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " Jun 15 09:20:04.365710 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.365677 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-bound-sa-token\") pod \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " Jun 15 09:20:04.365906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.365746 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-ca-trust-extracted\") pod \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\" (UID: \"0690cee7-ba7d-4990-9e23-4d86f39cdd5c\") " Jun 15 09:20:04.366387 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.365966 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0690cee7-ba7d-4990-9e23-4d86f39cdd5c" (UID: "0690cee7-ba7d-4990-9e23-4d86f39cdd5c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 09:20:04.366387 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.366350 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0690cee7-ba7d-4990-9e23-4d86f39cdd5c" (UID: "0690cee7-ba7d-4990-9e23-4d86f39cdd5c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 09:20:04.368743 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.368695 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0690cee7-ba7d-4990-9e23-4d86f39cdd5c" (UID: "0690cee7-ba7d-4990-9e23-4d86f39cdd5c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 09:20:04.368863 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.368737 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-kube-api-access-hmxw7" (OuterVolumeSpecName: "kube-api-access-hmxw7") pod "0690cee7-ba7d-4990-9e23-4d86f39cdd5c" (UID: "0690cee7-ba7d-4990-9e23-4d86f39cdd5c"). InnerVolumeSpecName "kube-api-access-hmxw7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 09:20:04.370141 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.370104 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0690cee7-ba7d-4990-9e23-4d86f39cdd5c" (UID: "0690cee7-ba7d-4990-9e23-4d86f39cdd5c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 09:20:04.370141 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.370114 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0690cee7-ba7d-4990-9e23-4d86f39cdd5c" (UID: "0690cee7-ba7d-4990-9e23-4d86f39cdd5c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 09:20:04.370368 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.370346 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0690cee7-ba7d-4990-9e23-4d86f39cdd5c" (UID: "0690cee7-ba7d-4990-9e23-4d86f39cdd5c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 09:20:04.376918 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.376821 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0690cee7-ba7d-4990-9e23-4d86f39cdd5c" (UID: "0690cee7-ba7d-4990-9e23-4d86f39cdd5c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 09:20:04.377449 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.377431 2562 scope.go:117] "RemoveContainer" containerID="33836c7cd7fa6c32ed8dcec04bb88cea8c893157d882493da0a069857a6b059a" Jun 15 09:20:04.377753 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:20:04.377732 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33836c7cd7fa6c32ed8dcec04bb88cea8c893157d882493da0a069857a6b059a\": container with ID starting with 33836c7cd7fa6c32ed8dcec04bb88cea8c893157d882493da0a069857a6b059a not found: ID does not exist" containerID="33836c7cd7fa6c32ed8dcec04bb88cea8c893157d882493da0a069857a6b059a" Jun 15 09:20:04.377853 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.377761 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33836c7cd7fa6c32ed8dcec04bb88cea8c893157d882493da0a069857a6b059a"} err="failed to get container status \"33836c7cd7fa6c32ed8dcec04bb88cea8c893157d882493da0a069857a6b059a\": rpc error: code = NotFound desc = could not find container \"33836c7cd7fa6c32ed8dcec04bb88cea8c893157d882493da0a069857a6b059a\": container with ID starting with 33836c7cd7fa6c32ed8dcec04bb88cea8c893157d882493da0a069857a6b059a not found: ID does not exist" Jun 15 09:20:04.467278 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.467231 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmxw7\" (UniqueName: \"kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-kube-api-access-hmxw7\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:20:04.467278 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.467273 2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-bound-sa-token\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:20:04.467278 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.467282 2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-ca-trust-extracted\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:20:04.467278 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.467295 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-trusted-ca\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:20:04.467542 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.467304 2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-installation-pull-secrets\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:20:04.467542 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.467312 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-tls\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:20:04.467542 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.467320 2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-registry-certificates\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:20:04.467542 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.467330 2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0690cee7-ba7d-4990-9e23-4d86f39cdd5c-image-registry-private-configuration\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:20:04.685458 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.685430 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66c98d86-pkr99"] Jun 15 09:20:04.688793 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:04.688768 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-66c98d86-pkr99"] Jun 15 09:20:05.318020 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:05.317984 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:05.418282 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:05.418245 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:05.934480 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:05.934396 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0690cee7-ba7d-4990-9e23-4d86f39cdd5c" path="/var/lib/kubelet/pods/0690cee7-ba7d-4990-9e23-4d86f39cdd5c/volumes" Jun 15 09:20:06.376635 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:06.376602 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7aab4282-57ea-4f2b-a4f4-008802f60066","Type":"ContainerStarted","Data":"b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e"} Jun 15 09:20:06.377053 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:06.376692 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7aab4282-57ea-4f2b-a4f4-008802f60066","Type":"ContainerStarted","Data":"b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb"} Jun 15 09:20:06.377053 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:06.376720 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7aab4282-57ea-4f2b-a4f4-008802f60066","Type":"ContainerStarted","Data":"bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a"} Jun 15 09:20:06.377053 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:06.376746 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7aab4282-57ea-4f2b-a4f4-008802f60066","Type":"ContainerStarted","Data":"c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787"} Jun 15 09:20:06.382694 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:06.382665 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:06.407666 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:06.407602 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.248335622 podStartE2EDuration="9.407588819s" totalImagesPullingTime="6.135709129s" totalInitContainerRuntime="23.544068ms" isStatefulPod=false podCreationTimestamp="2026-06-15 09:19:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:20:06.405792796 +0000 UTC m=+90.969215122" watchObservedRunningTime="2026-06-15 09:20:06.407588819 +0000 UTC m=+90.971011123" Jun 15 09:20:06.444078 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:06.444050 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:08.114947 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:08.114904 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:08.213430 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:08.213397 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:08.261235 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:08.261207 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:20:09.548130 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:09.548095 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:09.597184 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:09.597151 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:09.739798 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:09.739768 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:09.791954 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:09.791921 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:11.195337 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:11.195304 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:13.942995 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:13.942954 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:14.039493 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:14.039463 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:14.138987 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:14.138952 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:14.685218 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:14.685187 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:20.312202 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:20.312166 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:20.362144 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:20.362115 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:21.310412 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:21.310387 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-cdmf7" Jun 15 09:20:21.666219 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:21.666192 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gwnsl_22891e8c-12f1-48c2-910f-5015573cdad8/serve-healthcheck-canary/0.log" Jun 15 09:20:22.669770 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:22.669735 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:22.771157 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:22.771129 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:22.823609 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:22.823583 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:24.480145 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:24.480098 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:24.584498 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:24.584462 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:24.684740 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:24.684711 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:35.321763 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:35.321690 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:35.412542 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:35.412507 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:20:58.260658 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:58.260626 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:20:58.279742 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:58.279712 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:20:58.567162 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:20:58.567138 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:07.812901 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:07.812824 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:21:07.863730 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:07.863699 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:21:16.277571 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.277525 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 15 09:21:16.278032 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.277818 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="prometheus" containerID="cri-o://e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689" gracePeriod=600 Jun 15 09:21:16.278032 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.277865 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="kube-rbac-proxy" containerID="cri-o://b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb" gracePeriod=600 Jun 15 09:21:16.278032 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.277907 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="config-reloader" containerID="cri-o://858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155" gracePeriod=600 Jun 15 09:21:16.278032 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.277915 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="thanos-sidecar" containerID="cri-o://c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787" gracePeriod=600 Jun 15 09:21:16.278032 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.277963 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="kube-rbac-proxy-web" containerID="cri-o://bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a" gracePeriod=600 Jun 15 09:21:16.278032 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.277919 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="kube-rbac-proxy-thanos" containerID="cri-o://b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e" gracePeriod=600 Jun 15 09:21:16.542062 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.542038 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:16.617052 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.617019 2562 generic.go:356] "Generic (PLEG): container finished" podID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerID="b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e" exitCode=0 Jun 15 09:21:16.617052 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.617049 2562 generic.go:356] "Generic (PLEG): container finished" podID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerID="b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb" exitCode=0 Jun 15 09:21:16.617052 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.617061 2562 generic.go:356] "Generic (PLEG): container finished" podID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerID="bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a" exitCode=0 Jun 15 09:21:16.617364 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.617069 2562 generic.go:356] "Generic (PLEG): container finished" podID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerID="c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787" exitCode=0 Jun 15 09:21:16.617364 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.617063 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7aab4282-57ea-4f2b-a4f4-008802f60066","Type":"ContainerDied","Data":"b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e"} Jun 15 09:21:16.617364 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.617119 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:16.617364 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.617077 2562 generic.go:356] "Generic (PLEG): container finished" podID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerID="858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155" exitCode=0 Jun 15 09:21:16.617364 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.617160 2562 generic.go:356] "Generic (PLEG): container finished" podID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerID="e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689" exitCode=0 Jun 15 09:21:16.617364 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.617163 2562 scope.go:117] "RemoveContainer" containerID="b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e" Jun 15 09:21:16.617364 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.617152 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7aab4282-57ea-4f2b-a4f4-008802f60066","Type":"ContainerDied","Data":"b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb"} Jun 15 09:21:16.617364 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.617230 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7aab4282-57ea-4f2b-a4f4-008802f60066","Type":"ContainerDied","Data":"bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a"} Jun 15 09:21:16.617364 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.617262 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7aab4282-57ea-4f2b-a4f4-008802f60066","Type":"ContainerDied","Data":"c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787"} Jun 15 09:21:16.617364 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.617308 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7aab4282-57ea-4f2b-a4f4-008802f60066","Type":"ContainerDied","Data":"858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155"} Jun 15 09:21:16.617364 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.617341 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7aab4282-57ea-4f2b-a4f4-008802f60066","Type":"ContainerDied","Data":"e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689"} Jun 15 09:21:16.617808 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.617380 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7aab4282-57ea-4f2b-a4f4-008802f60066","Type":"ContainerDied","Data":"cb476189d805e6172cd33b3338cb02042d98aca4a0048fa55ed7510d6fce6460"} Jun 15 09:21:16.625162 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.625147 2562 scope.go:117] "RemoveContainer" containerID="b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb" Jun 15 09:21:16.631591 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.631565 2562 scope.go:117] "RemoveContainer" containerID="bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a" Jun 15 09:21:16.638255 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.638236 2562 scope.go:117] "RemoveContainer" containerID="c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787" Jun 15 09:21:16.646221 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.646203 2562 scope.go:117] "RemoveContainer" containerID="858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155" Jun 15 09:21:16.652780 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.652762 2562 scope.go:117] "RemoveContainer" containerID="e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689" Jun 15 09:21:16.659640 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.659617 2562 scope.go:117] "RemoveContainer" containerID="a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840" Jun 15 09:21:16.666106 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.666088 2562 scope.go:117] "RemoveContainer" containerID="b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e" Jun 15 09:21:16.666362 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:21:16.666343 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e\": container with ID starting with b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e not found: ID does not exist" containerID="b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e" Jun 15 09:21:16.666434 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.666375 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e"} err="failed to get container status \"b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e\": rpc error: code = NotFound desc = could not find container \"b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e\": container with ID starting with b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e not found: ID does not exist" Jun 15 09:21:16.666434 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.666402 2562 scope.go:117] "RemoveContainer" containerID="b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb" Jun 15 09:21:16.666647 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:21:16.666627 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb\": container with ID starting with b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb not found: ID does not exist" containerID="b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb" Jun 15 09:21:16.666685 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.666655 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb"} err="failed to get container status \"b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb\": rpc error: code = NotFound desc = could not find container \"b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb\": container with ID starting with b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb not found: ID does not exist" Jun 15 09:21:16.666685 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.666671 2562 scope.go:117] "RemoveContainer" containerID="bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a" Jun 15 09:21:16.666909 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:21:16.666879 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a\": container with ID starting with bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a not found: ID does not exist" containerID="bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a" Jun 15 09:21:16.667013 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.666907 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a"} err="failed to get container status \"bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a\": rpc error: code = NotFound desc = could not find container \"bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a\": container with ID starting with bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a not found: ID does not exist" Jun 15 09:21:16.667013 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.666927 2562 scope.go:117] "RemoveContainer" containerID="c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787" Jun 15 09:21:16.667156 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:21:16.667140 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787\": container with ID starting with c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787 not found: ID does not exist" containerID="c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787" Jun 15 09:21:16.667194 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.667160 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787"} err="failed to get container status \"c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787\": rpc error: code = NotFound desc = could not find container \"c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787\": container with ID starting with c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787 not found: ID does not exist" Jun 15 09:21:16.667194 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.667173 2562 scope.go:117] "RemoveContainer" containerID="858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155" Jun 15 09:21:16.667406 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:21:16.667389 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155\": container with ID starting with 858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155 not found: ID does not exist" containerID="858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155" Jun 15 09:21:16.667450 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.667411 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155"} err="failed to get container status \"858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155\": rpc error: code = NotFound desc = could not find container \"858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155\": container with ID starting with 858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155 not found: ID does not exist" Jun 15 09:21:16.667450 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.667424 2562 scope.go:117] "RemoveContainer" containerID="e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689" Jun 15 09:21:16.667628 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:21:16.667614 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689\": container with ID starting with e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689 not found: ID does not exist" containerID="e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689" Jun 15 09:21:16.667672 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.667630 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689"} err="failed to get container status \"e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689\": rpc error: code = NotFound desc = could not find container \"e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689\": container with ID starting with e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689 not found: ID does not exist" Jun 15 09:21:16.667672 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.667642 2562 scope.go:117] "RemoveContainer" containerID="a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840" Jun 15 09:21:16.667845 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:21:16.667815 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840\": container with ID starting with a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840 not found: ID does not exist" containerID="a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840" Jun 15 09:21:16.667893 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.667870 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840"} err="failed to get container status \"a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840\": rpc error: code = NotFound desc = could not find container \"a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840\": container with ID starting with a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840 not found: ID does not exist" Jun 15 09:21:16.667930 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.667892 2562 scope.go:117] "RemoveContainer" containerID="b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e" Jun 15 09:21:16.668144 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.668125 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e"} err="failed to get container status \"b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e\": rpc error: code = NotFound desc = could not find container \"b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e\": container with ID starting with b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e not found: ID does not exist" Jun 15 09:21:16.668188 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.668145 2562 scope.go:117] "RemoveContainer" containerID="b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb" Jun 15 09:21:16.668341 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.668323 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb"} err="failed to get container status \"b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb\": rpc error: code = NotFound desc = could not find container \"b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb\": container with ID starting with b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb not found: ID does not exist" Jun 15 09:21:16.668395 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.668341 2562 scope.go:117] "RemoveContainer" containerID="bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a" Jun 15 09:21:16.668513 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.668499 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a"} err="failed to get container status \"bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a\": rpc error: code = NotFound desc = could not find container \"bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a\": container with ID starting with bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a not found: ID does not exist" Jun 15 09:21:16.668563 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.668513 2562 scope.go:117] "RemoveContainer" containerID="c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787" Jun 15 09:21:16.668679 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.668660 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787"} err="failed to get container status \"c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787\": rpc error: code = NotFound desc = could not find container \"c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787\": container with ID starting with c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787 not found: ID does not exist" Jun 15 09:21:16.668752 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.668681 2562 scope.go:117] "RemoveContainer" containerID="858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155" Jun 15 09:21:16.668896 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.668881 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155"} err="failed to get container status \"858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155\": rpc error: code = NotFound desc = could not find container \"858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155\": container with ID starting with 858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155 not found: ID does not exist" Jun 15 09:21:16.668954 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.668896 2562 scope.go:117] "RemoveContainer" containerID="e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689" Jun 15 09:21:16.669097 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.669072 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689"} err="failed to get container status \"e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689\": rpc error: code = NotFound desc = could not find container \"e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689\": container with ID starting with e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689 not found: ID does not exist" Jun 15 09:21:16.669144 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.669098 2562 scope.go:117] "RemoveContainer" containerID="a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840" Jun 15 09:21:16.669293 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.669274 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840"} err="failed to get container status \"a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840\": rpc error: code = NotFound desc = could not find container \"a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840\": container with ID starting with a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840 not found: ID does not exist" Jun 15 09:21:16.669342 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.669293 2562 scope.go:117] "RemoveContainer" containerID="b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e" Jun 15 09:21:16.669504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.669489 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e"} err="failed to get container status \"b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e\": rpc error: code = NotFound desc = could not find container \"b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e\": container with ID starting with b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e not found: ID does not exist" Jun 15 09:21:16.669555 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.669506 2562 scope.go:117] "RemoveContainer" containerID="b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb" Jun 15 09:21:16.669702 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.669678 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb"} err="failed to get container status \"b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb\": rpc error: code = NotFound desc = could not find container \"b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb\": container with ID starting with b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb not found: ID does not exist" Jun 15 09:21:16.669741 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.669704 2562 scope.go:117] "RemoveContainer" containerID="bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a" Jun 15 09:21:16.669941 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.669923 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a"} err="failed to get container status \"bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a\": rpc error: code = NotFound desc = could not find container \"bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a\": container with ID starting with bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a not found: ID does not exist" Jun 15 09:21:16.669991 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.669942 2562 scope.go:117] "RemoveContainer" containerID="c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787" Jun 15 09:21:16.670156 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.670140 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787"} err="failed to get container status \"c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787\": rpc error: code = NotFound desc = could not find container \"c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787\": container with ID starting with c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787 not found: ID does not exist" Jun 15 09:21:16.670200 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.670157 2562 scope.go:117] "RemoveContainer" containerID="858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155" Jun 15 09:21:16.670344 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.670325 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155"} err="failed to get container status \"858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155\": rpc error: code = NotFound desc = could not find container \"858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155\": container with ID starting with 858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155 not found: ID does not exist" Jun 15 09:21:16.670384 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.670346 2562 scope.go:117] "RemoveContainer" containerID="e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689" Jun 15 09:21:16.670554 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.670536 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689"} err="failed to get container status \"e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689\": rpc error: code = NotFound desc = could not find container \"e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689\": container with ID starting with e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689 not found: ID does not exist" Jun 15 09:21:16.670601 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.670555 2562 scope.go:117] "RemoveContainer" containerID="a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840" Jun 15 09:21:16.670765 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.670745 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840"} err="failed to get container status \"a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840\": rpc error: code = NotFound desc = could not find container \"a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840\": container with ID starting with a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840 not found: ID does not exist" Jun 15 09:21:16.670806 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.670766 2562 scope.go:117] "RemoveContainer" containerID="b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e" Jun 15 09:21:16.671058 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.671041 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e"} err="failed to get container status \"b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e\": rpc error: code = NotFound desc = could not find container \"b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e\": container with ID starting with b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e not found: ID does not exist" Jun 15 09:21:16.671107 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.671057 2562 scope.go:117] "RemoveContainer" containerID="b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb" Jun 15 09:21:16.671244 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.671227 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb"} err="failed to get container status \"b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb\": rpc error: code = NotFound desc = could not find container \"b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb\": container with ID starting with b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb not found: ID does not exist" Jun 15 09:21:16.671285 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.671245 2562 scope.go:117] "RemoveContainer" containerID="bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a" Jun 15 09:21:16.671449 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.671428 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a"} err="failed to get container status \"bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a\": rpc error: code = NotFound desc = could not find container \"bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a\": container with ID starting with bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a not found: ID does not exist" Jun 15 09:21:16.671490 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.671450 2562 scope.go:117] "RemoveContainer" containerID="c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787" Jun 15 09:21:16.671660 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.671645 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787"} err="failed to get container status \"c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787\": rpc error: code = NotFound desc = could not find container \"c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787\": container with ID starting with c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787 not found: ID does not exist" Jun 15 09:21:16.671707 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.671660 2562 scope.go:117] "RemoveContainer" containerID="858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155" Jun 15 09:21:16.671831 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.671809 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155"} err="failed to get container status \"858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155\": rpc error: code = NotFound desc = could not find container \"858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155\": container with ID starting with 858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155 not found: ID does not exist" Jun 15 09:21:16.671885 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.671844 2562 scope.go:117] "RemoveContainer" containerID="e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689" Jun 15 09:21:16.672037 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.672020 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689"} err="failed to get container status \"e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689\": rpc error: code = NotFound desc = could not find container \"e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689\": container with ID starting with e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689 not found: ID does not exist" Jun 15 09:21:16.672077 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.672036 2562 scope.go:117] "RemoveContainer" containerID="a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840" Jun 15 09:21:16.672226 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.672208 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840"} err="failed to get container status \"a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840\": rpc error: code = NotFound desc = could not find container \"a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840\": container with ID starting with a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840 not found: ID does not exist" Jun 15 09:21:16.672271 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.672227 2562 scope.go:117] "RemoveContainer" containerID="b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e" Jun 15 09:21:16.672447 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.672426 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e"} err="failed to get container status \"b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e\": rpc error: code = NotFound desc = could not find container \"b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e\": container with ID starting with b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e not found: ID does not exist" Jun 15 09:21:16.672516 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.672449 2562 scope.go:117] "RemoveContainer" containerID="b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb" Jun 15 09:21:16.672673 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.672658 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb"} err="failed to get container status \"b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb\": rpc error: code = NotFound desc = could not find container \"b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb\": container with ID starting with b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb not found: ID does not exist" Jun 15 09:21:16.672724 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.672673 2562 scope.go:117] "RemoveContainer" containerID="bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a" Jun 15 09:21:16.672882 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.672863 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a"} err="failed to get container status \"bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a\": rpc error: code = NotFound desc = could not find container \"bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a\": container with ID starting with bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a not found: ID does not exist" Jun 15 09:21:16.672939 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.672882 2562 scope.go:117] "RemoveContainer" containerID="c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787" Jun 15 09:21:16.673091 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.673074 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787"} err="failed to get container status \"c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787\": rpc error: code = NotFound desc = could not find container \"c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787\": container with ID starting with c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787 not found: ID does not exist" Jun 15 09:21:16.673154 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.673093 2562 scope.go:117] "RemoveContainer" containerID="858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155" Jun 15 09:21:16.673327 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.673308 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155"} err="failed to get container status \"858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155\": rpc error: code = NotFound desc = could not find container \"858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155\": container with ID starting with 858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155 not found: ID does not exist" Jun 15 09:21:16.673376 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.673328 2562 scope.go:117] "RemoveContainer" containerID="e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689" Jun 15 09:21:16.673523 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.673507 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689"} err="failed to get container status \"e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689\": rpc error: code = NotFound desc = could not find container \"e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689\": container with ID starting with e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689 not found: ID does not exist" Jun 15 09:21:16.673576 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.673523 2562 scope.go:117] "RemoveContainer" containerID="a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840" Jun 15 09:21:16.673706 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.673691 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840"} err="failed to get container status \"a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840\": rpc error: code = NotFound desc = could not find container \"a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840\": container with ID starting with a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840 not found: ID does not exist" Jun 15 09:21:16.673747 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.673705 2562 scope.go:117] "RemoveContainer" containerID="b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e" Jun 15 09:21:16.673909 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.673892 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e"} err="failed to get container status \"b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e\": rpc error: code = NotFound desc = could not find container \"b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e\": container with ID starting with b3dddf85809bd991b4d7146b87076001c9470299cd4ac87df962424f052d632e not found: ID does not exist" Jun 15 09:21:16.673973 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.673912 2562 scope.go:117] "RemoveContainer" containerID="b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb" Jun 15 09:21:16.674122 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.674105 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb"} err="failed to get container status \"b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb\": rpc error: code = NotFound desc = could not find container \"b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb\": container with ID starting with b503691293638916181155f2bbe8072dcaf21f70fbdfa449ce37ce7fc42f00cb not found: ID does not exist" Jun 15 09:21:16.674166 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.674123 2562 scope.go:117] "RemoveContainer" containerID="bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a" Jun 15 09:21:16.674342 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.674325 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a"} err="failed to get container status \"bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a\": rpc error: code = NotFound desc = could not find container \"bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a\": container with ID starting with bc91347c2a1f7d58fac470631bd18f754abda8b11232cca43038202428b9fa2a not found: ID does not exist" Jun 15 09:21:16.674408 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.674344 2562 scope.go:117] "RemoveContainer" containerID="c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787" Jun 15 09:21:16.674574 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.674554 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787"} err="failed to get container status \"c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787\": rpc error: code = NotFound desc = could not find container \"c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787\": container with ID starting with c21bf53b731697e78f0bc4498b4e004ca1d0f081f4114098b3e15d7054e6a787 not found: ID does not exist" Jun 15 09:21:16.674574 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.674572 2562 scope.go:117] "RemoveContainer" containerID="858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155" Jun 15 09:21:16.674793 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.674775 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155"} err="failed to get container status \"858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155\": rpc error: code = NotFound desc = could not find container \"858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155\": container with ID starting with 858d94835f510285ef4613071c44c72a82d81c9e003936ca5fc2d5f771a42155 not found: ID does not exist" Jun 15 09:21:16.674898 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.674794 2562 scope.go:117] "RemoveContainer" containerID="e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689" Jun 15 09:21:16.675031 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.675015 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689"} err="failed to get container status \"e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689\": rpc error: code = NotFound desc = could not find container \"e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689\": container with ID starting with e9efaee5ce324d2d9718607f13d65982c65f82f169c8889fc22df46e84019689 not found: ID does not exist" Jun 15 09:21:16.675082 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.675031 2562 scope.go:117] "RemoveContainer" containerID="a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840" Jun 15 09:21:16.675230 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.675215 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840"} err="failed to get container status \"a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840\": rpc error: code = NotFound desc = could not find container \"a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840\": container with ID starting with a14c1f2ba0e7613c692ef50c4b7fff1bca03ebd4e001cb0799c0cf2d2aa24840 not found: ID does not exist" Jun 15 09:21:16.692522 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.692501 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"7aab4282-57ea-4f2b-a4f4-008802f60066\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " Jun 15 09:21:16.692591 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.692529 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-configmap-kubelet-serving-ca-bundle\") pod \"7aab4282-57ea-4f2b-a4f4-008802f60066\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " Jun 15 09:21:16.692591 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.692546 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-rulefiles-0\") pod \"7aab4282-57ea-4f2b-a4f4-008802f60066\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " Jun 15 09:21:16.692591 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.692562 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-configmap-serving-certs-ca-bundle\") pod \"7aab4282-57ea-4f2b-a4f4-008802f60066\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " Jun 15 09:21:16.692591 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.692586 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-config\") pod \"7aab4282-57ea-4f2b-a4f4-008802f60066\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " Jun 15 09:21:16.692780 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.692607 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7aab4282-57ea-4f2b-a4f4-008802f60066-config-out\") pod \"7aab4282-57ea-4f2b-a4f4-008802f60066\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " Jun 15 09:21:16.692780 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.692631 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7aab4282-57ea-4f2b-a4f4-008802f60066-tls-assets\") pod \"7aab4282-57ea-4f2b-a4f4-008802f60066\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " Jun 15 09:21:16.692780 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.692648 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"7aab4282-57ea-4f2b-a4f4-008802f60066\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " Jun 15 09:21:16.692780 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.692667 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8p5t9\" (UniqueName: \"kubernetes.io/projected/7aab4282-57ea-4f2b-a4f4-008802f60066-kube-api-access-8p5t9\") pod \"7aab4282-57ea-4f2b-a4f4-008802f60066\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " Jun 15 09:21:16.692780 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.692732 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-trusted-ca-bundle\") pod \"7aab4282-57ea-4f2b-a4f4-008802f60066\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " Jun 15 09:21:16.692780 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.692759 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-metrics-client-certs\") pod \"7aab4282-57ea-4f2b-a4f4-008802f60066\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " Jun 15 09:21:16.692780 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.692779 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-web-config\") pod \"7aab4282-57ea-4f2b-a4f4-008802f60066\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " Jun 15 09:21:16.693129 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.692808 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-prometheus-k8s-tls\") pod \"7aab4282-57ea-4f2b-a4f4-008802f60066\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " Jun 15 09:21:16.693129 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.692857 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-rulefiles-1\") pod \"7aab4282-57ea-4f2b-a4f4-008802f60066\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " Jun 15 09:21:16.693129 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.692883 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-rulefiles-2\") pod \"7aab4282-57ea-4f2b-a4f4-008802f60066\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " Jun 15 09:21:16.693129 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.692928 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-db\") pod \"7aab4282-57ea-4f2b-a4f4-008802f60066\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " Jun 15 09:21:16.693129 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.692958 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-kube-rbac-proxy\") pod \"7aab4282-57ea-4f2b-a4f4-008802f60066\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " Jun 15 09:21:16.693129 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.692971 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "7aab4282-57ea-4f2b-a4f4-008802f60066" (UID: "7aab4282-57ea-4f2b-a4f4-008802f60066"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 09:21:16.693129 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.692994 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-thanos-prometheus-http-client-file\") pod \"7aab4282-57ea-4f2b-a4f4-008802f60066\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " Jun 15 09:21:16.693129 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.693020 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-configmap-metrics-client-ca\") pod \"7aab4282-57ea-4f2b-a4f4-008802f60066\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " Jun 15 09:21:16.693129 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.693057 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-grpc-tls\") pod \"7aab4282-57ea-4f2b-a4f4-008802f60066\" (UID: \"7aab4282-57ea-4f2b-a4f4-008802f60066\") " Jun 15 09:21:16.693575 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.693263 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:21:16.694239 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.693805 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "7aab4282-57ea-4f2b-a4f4-008802f60066" (UID: "7aab4282-57ea-4f2b-a4f4-008802f60066"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 09:21:16.694239 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.693960 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "7aab4282-57ea-4f2b-a4f4-008802f60066" (UID: "7aab4282-57ea-4f2b-a4f4-008802f60066"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 09:21:16.694239 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.693986 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "7aab4282-57ea-4f2b-a4f4-008802f60066" (UID: "7aab4282-57ea-4f2b-a4f4-008802f60066"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 09:21:16.694239 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.694170 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "7aab4282-57ea-4f2b-a4f4-008802f60066" (UID: "7aab4282-57ea-4f2b-a4f4-008802f60066"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 09:21:16.694572 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.694371 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "7aab4282-57ea-4f2b-a4f4-008802f60066" (UID: "7aab4282-57ea-4f2b-a4f4-008802f60066"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 09:21:16.695167 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.694652 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "7aab4282-57ea-4f2b-a4f4-008802f60066" (UID: "7aab4282-57ea-4f2b-a4f4-008802f60066"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 09:21:16.695167 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.695136 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "7aab4282-57ea-4f2b-a4f4-008802f60066" (UID: "7aab4282-57ea-4f2b-a4f4-008802f60066"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 09:21:16.696102 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.696076 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "7aab4282-57ea-4f2b-a4f4-008802f60066" (UID: "7aab4282-57ea-4f2b-a4f4-008802f60066"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 09:21:16.696432 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.696237 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "7aab4282-57ea-4f2b-a4f4-008802f60066" (UID: "7aab4282-57ea-4f2b-a4f4-008802f60066"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 09:21:16.696432 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.696400 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7aab4282-57ea-4f2b-a4f4-008802f60066-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "7aab4282-57ea-4f2b-a4f4-008802f60066" (UID: "7aab4282-57ea-4f2b-a4f4-008802f60066"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 09:21:16.696432 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.696395 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "7aab4282-57ea-4f2b-a4f4-008802f60066" (UID: "7aab4282-57ea-4f2b-a4f4-008802f60066"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 09:21:16.696432 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.696408 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-config" (OuterVolumeSpecName: "config") pod "7aab4282-57ea-4f2b-a4f4-008802f60066" (UID: "7aab4282-57ea-4f2b-a4f4-008802f60066"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 09:21:16.696782 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.696761 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "7aab4282-57ea-4f2b-a4f4-008802f60066" (UID: "7aab4282-57ea-4f2b-a4f4-008802f60066"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 09:21:16.697206 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.697186 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7aab4282-57ea-4f2b-a4f4-008802f60066-kube-api-access-8p5t9" (OuterVolumeSpecName: "kube-api-access-8p5t9") pod "7aab4282-57ea-4f2b-a4f4-008802f60066" (UID: "7aab4282-57ea-4f2b-a4f4-008802f60066"). InnerVolumeSpecName "kube-api-access-8p5t9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 09:21:16.697568 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.697550 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7aab4282-57ea-4f2b-a4f4-008802f60066-config-out" (OuterVolumeSpecName: "config-out") pod "7aab4282-57ea-4f2b-a4f4-008802f60066" (UID: "7aab4282-57ea-4f2b-a4f4-008802f60066"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 09:21:16.697752 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.697735 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "7aab4282-57ea-4f2b-a4f4-008802f60066" (UID: "7aab4282-57ea-4f2b-a4f4-008802f60066"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 09:21:16.697878 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.697862 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "7aab4282-57ea-4f2b-a4f4-008802f60066" (UID: "7aab4282-57ea-4f2b-a4f4-008802f60066"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 09:21:16.697968 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.697947 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "7aab4282-57ea-4f2b-a4f4-008802f60066" (UID: "7aab4282-57ea-4f2b-a4f4-008802f60066"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 09:21:16.706298 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.706278 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-web-config" (OuterVolumeSpecName: "web-config") pod "7aab4282-57ea-4f2b-a4f4-008802f60066" (UID: "7aab4282-57ea-4f2b-a4f4-008802f60066"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 09:21:16.794420 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.794328 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:21:16.794420 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.794365 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:21:16.794420 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.794377 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:21:16.794420 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.794388 2562 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-config\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:21:16.794420 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.794397 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7aab4282-57ea-4f2b-a4f4-008802f60066-config-out\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:21:16.794420 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.794405 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7aab4282-57ea-4f2b-a4f4-008802f60066-tls-assets\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:21:16.794420 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.794412 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:21:16.794420 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.794421 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p5t9\" (UniqueName: \"kubernetes.io/projected/7aab4282-57ea-4f2b-a4f4-008802f60066-kube-api-access-8p5t9\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:21:16.794420 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.794429 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:21:16.794874 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.794438 2562 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-metrics-client-certs\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:21:16.794874 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.794447 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-web-config\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:21:16.794874 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.794458 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:21:16.794874 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.794466 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:21:16.794874 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.794474 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:21:16.794874 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.794484 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7aab4282-57ea-4f2b-a4f4-008802f60066-prometheus-k8s-db\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:21:16.794874 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.794494 2562 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-kube-rbac-proxy\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:21:16.794874 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.794502 2562 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:21:16.794874 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.794511 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7aab4282-57ea-4f2b-a4f4-008802f60066-configmap-metrics-client-ca\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:21:16.794874 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.794519 2562 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7aab4282-57ea-4f2b-a4f4-008802f60066-secret-grpc-tls\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:21:16.940704 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.940664 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 15 09:21:16.944801 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.944779 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 15 09:21:16.968346 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968315 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 15 09:21:16.968544 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968530 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0690cee7-ba7d-4990-9e23-4d86f39cdd5c" containerName="registry" Jun 15 09:21:16.968590 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968545 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0690cee7-ba7d-4990-9e23-4d86f39cdd5c" containerName="registry" Jun 15 09:21:16.968590 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968556 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="prometheus" Jun 15 09:21:16.968590 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968561 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="prometheus" Jun 15 09:21:16.968590 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968567 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="config-reloader" Jun 15 09:21:16.968590 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968573 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="config-reloader" Jun 15 09:21:16.968590 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968578 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="thanos-sidecar" Jun 15 09:21:16.968590 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968583 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="thanos-sidecar" Jun 15 09:21:16.968590 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968588 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="kube-rbac-proxy-thanos" Jun 15 09:21:16.968590 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968594 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="kube-rbac-proxy-thanos" Jun 15 09:21:16.968900 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968604 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="kube-rbac-proxy-web" Jun 15 09:21:16.968900 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968609 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="kube-rbac-proxy-web" Jun 15 09:21:16.968900 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968616 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="kube-rbac-proxy" Jun 15 09:21:16.968900 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968621 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="kube-rbac-proxy" Jun 15 09:21:16.968900 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968630 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="init-config-reloader" Jun 15 09:21:16.968900 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968636 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="init-config-reloader" Jun 15 09:21:16.968900 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968673 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="prometheus" Jun 15 09:21:16.968900 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968680 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="thanos-sidecar" Jun 15 09:21:16.968900 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968686 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="kube-rbac-proxy-thanos" Jun 15 09:21:16.968900 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968692 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="config-reloader" Jun 15 09:21:16.968900 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968697 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="kube-rbac-proxy-web" Jun 15 09:21:16.968900 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968703 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0690cee7-ba7d-4990-9e23-4d86f39cdd5c" containerName="registry" Jun 15 09:21:16.968900 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.968710 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" containerName="kube-rbac-proxy" Jun 15 09:21:16.971989 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.971974 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:16.974787 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.974641 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 15 09:21:16.974787 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.974651 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 15 09:21:16.974787 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.974700 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 15 09:21:16.974787 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.974654 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 15 09:21:16.974787 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.974646 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-f5dg2\"" Jun 15 09:21:16.974787 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.974747 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 15 09:21:16.974787 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.974730 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 15 09:21:16.975144 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.975037 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7odhctu18u789\"" Jun 15 09:21:16.975519 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.975500 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 15 09:21:16.975601 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.975511 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 15 09:21:16.976436 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.976412 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 15 09:21:16.976530 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.976444 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 15 09:21:16.976686 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.976626 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 15 09:21:16.976686 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.976657 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 15 09:21:16.977022 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.976914 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 15 09:21:16.977940 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.977916 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 15 09:21:16.984605 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.982233 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 15 09:21:16.987557 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:16.987535 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 15 09:21:17.097187 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.097153 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38d75778-28a1-4c29-bcb2-46470d15cae4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.097344 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.097196 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38d75778-28a1-4c29-bcb2-46470d15cae4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.097344 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.097224 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38d75778-28a1-4c29-bcb2-46470d15cae4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.097344 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.097274 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.097344 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.097299 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.097344 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.097316 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38d75778-28a1-4c29-bcb2-46470d15cae4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.097344 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.097334 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38d75778-28a1-4c29-bcb2-46470d15cae4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.097608 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.097374 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-web-config\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.097608 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.097421 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5bsn\" (UniqueName: \"kubernetes.io/projected/38d75778-28a1-4c29-bcb2-46470d15cae4-kube-api-access-p5bsn\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.097608 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.097451 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38d75778-28a1-4c29-bcb2-46470d15cae4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.097608 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.097471 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-config\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.097608 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.097499 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.097608 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.097518 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38d75778-28a1-4c29-bcb2-46470d15cae4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.097608 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.097534 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38d75778-28a1-4c29-bcb2-46470d15cae4-config-out\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.097608 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.097561 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.097906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.097629 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.097906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.097667 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.097906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.097689 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38d75778-28a1-4c29-bcb2-46470d15cae4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.097906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.097707 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.097906 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.097725 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38d75778-28a1-4c29-bcb2-46470d15cae4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.198118 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.198079 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38d75778-28a1-4c29-bcb2-46470d15cae4-config-out\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.198118 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.198119 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.198360 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.198142 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.198360 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.198166 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.198360 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.198193 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38d75778-28a1-4c29-bcb2-46470d15cae4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.198360 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.198219 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.198551 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.198354 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38d75778-28a1-4c29-bcb2-46470d15cae4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.198551 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.198402 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38d75778-28a1-4c29-bcb2-46470d15cae4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.198551 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.198460 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38d75778-28a1-4c29-bcb2-46470d15cae4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.198551 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.198497 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38d75778-28a1-4c29-bcb2-46470d15cae4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.198551 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.198548 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.198797 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.198571 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.198797 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.198607 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38d75778-28a1-4c29-bcb2-46470d15cae4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.198797 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.198636 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38d75778-28a1-4c29-bcb2-46470d15cae4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.198797 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.198659 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-web-config\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.198797 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.198682 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5bsn\" (UniqueName: \"kubernetes.io/projected/38d75778-28a1-4c29-bcb2-46470d15cae4-kube-api-access-p5bsn\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.198797 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.198714 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38d75778-28a1-4c29-bcb2-46470d15cae4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.198797 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.198740 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-config\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.198797 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.198777 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.198797 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.198799 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38d75778-28a1-4c29-bcb2-46470d15cae4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.199257 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.199135 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38d75778-28a1-4c29-bcb2-46470d15cae4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.200024 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.199647 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38d75778-28a1-4c29-bcb2-46470d15cae4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.201038 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.200523 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38d75778-28a1-4c29-bcb2-46470d15cae4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.201400 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.201372 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38d75778-28a1-4c29-bcb2-46470d15cae4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.201500 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.201455 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38d75778-28a1-4c29-bcb2-46470d15cae4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.201897 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.201775 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38d75778-28a1-4c29-bcb2-46470d15cae4-config-out\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.202152 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.202123 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.202303 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.202279 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.202369 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.202333 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38d75778-28a1-4c29-bcb2-46470d15cae4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.202423 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.202376 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.202652 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.202631 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38d75778-28a1-4c29-bcb2-46470d15cae4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.202729 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.202654 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38d75778-28a1-4c29-bcb2-46470d15cae4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.203157 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.203133 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.203247 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.203184 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38d75778-28a1-4c29-bcb2-46470d15cae4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.203768 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.203740 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-config\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.203880 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.203787 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.204123 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.204106 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-web-config\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.204313 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.204297 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.204609 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.204592 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38d75778-28a1-4c29-bcb2-46470d15cae4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.208949 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.208928 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5bsn\" (UniqueName: \"kubernetes.io/projected/38d75778-28a1-4c29-bcb2-46470d15cae4-kube-api-access-p5bsn\") pod \"prometheus-k8s-0\" (UID: \"38d75778-28a1-4c29-bcb2-46470d15cae4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.283508 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.283472 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:17.425307 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.425276 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 15 09:21:17.428946 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:21:17.428912 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38d75778_28a1_4c29_bcb2_46470d15cae4.slice/crio-aef5352c7a6d1f1ab91dc76164d14fa65ef9d0a8d62df80f8077aac313248c0d WatchSource:0}: Error finding container aef5352c7a6d1f1ab91dc76164d14fa65ef9d0a8d62df80f8077aac313248c0d: Status 404 returned error can't find the container with id aef5352c7a6d1f1ab91dc76164d14fa65ef9d0a8d62df80f8077aac313248c0d Jun 15 09:21:17.623722 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.623635 2562 generic.go:356] "Generic (PLEG): container finished" podID="38d75778-28a1-4c29-bcb2-46470d15cae4" containerID="4e5b62f491f958d050f4ddf89ce736aa039ade6a781f6712f110d007ec69295d" exitCode=0 Jun 15 09:21:17.623722 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.623712 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38d75778-28a1-4c29-bcb2-46470d15cae4","Type":"ContainerDied","Data":"4e5b62f491f958d050f4ddf89ce736aa039ade6a781f6712f110d007ec69295d"} Jun 15 09:21:17.623922 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.623778 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38d75778-28a1-4c29-bcb2-46470d15cae4","Type":"ContainerStarted","Data":"aef5352c7a6d1f1ab91dc76164d14fa65ef9d0a8d62df80f8077aac313248c0d"} Jun 15 09:21:17.937664 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:17.937572 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7aab4282-57ea-4f2b-a4f4-008802f60066" path="/var/lib/kubelet/pods/7aab4282-57ea-4f2b-a4f4-008802f60066/volumes" Jun 15 09:21:18.630741 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:18.630702 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38d75778-28a1-4c29-bcb2-46470d15cae4","Type":"ContainerStarted","Data":"c89e18e1c05166ba7153a0f0a7e76918364c021762a8714e63739944d36176c6"} Jun 15 09:21:18.631126 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:18.630767 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38d75778-28a1-4c29-bcb2-46470d15cae4","Type":"ContainerStarted","Data":"3d71ed1e655a2b2947cd74774c2998e8346f1af0d620b1107707f433ad2ab25f"} Jun 15 09:21:18.631126 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:18.630792 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38d75778-28a1-4c29-bcb2-46470d15cae4","Type":"ContainerStarted","Data":"1c31cd0a7d93e65c3c7a9564ebfc7177489a435bda988637c85faab9b68e2164"} Jun 15 09:21:18.631126 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:18.630820 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38d75778-28a1-4c29-bcb2-46470d15cae4","Type":"ContainerStarted","Data":"814dd89d7bf748dc68062fc400aa7a04817ea9c91834905b37e7b1ee85108fd8"} Jun 15 09:21:18.631126 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:18.630863 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38d75778-28a1-4c29-bcb2-46470d15cae4","Type":"ContainerStarted","Data":"5f237ac8af6d6ed13ac97f7c0721f8973f7e0ddc2a646fdc1fa53aaca05751dc"} Jun 15 09:21:18.631126 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:18.630887 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38d75778-28a1-4c29-bcb2-46470d15cae4","Type":"ContainerStarted","Data":"5ff30e929adabb833a4891779e30c6fff79bbc7bac8479d0800ad8cf17e286c3"} Jun 15 09:21:18.657256 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:18.657205 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.638784846 podStartE2EDuration="2.657187225s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.402379ms" isStatefulPod=false podCreationTimestamp="2026-06-15 09:21:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:21:18.655265922 +0000 UTC m=+163.218688227" watchObservedRunningTime="2026-06-15 09:21:18.657187225 +0000 UTC m=+163.220609508" Jun 15 09:21:19.666133 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:19.666096 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:21:19.714366 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:19.714340 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:21:22.283939 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:22.283906 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:21:35.359099 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:35.359011 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:21:35.459484 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:35.459449 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:21:44.729258 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:44.729216 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:21:44.777091 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:44.777063 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:21:51.841916 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:51.841873 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:21:51.890556 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:51.890532 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:21:52.335356 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:52.335321 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:21:52.382169 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:52.382134 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:21:55.873804 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:55.873769 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:21:55.923504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:55.923478 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:21:55.974022 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:21:55.973988 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:22:17.284521 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:22:17.284486 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:22:17.299488 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:22:17.299463 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:22:17.846406 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:22:17.846370 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 09:22:19.124475 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:22:19.124438 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:22:19.223643 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:22:19.223614 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:22:24.936053 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:22:24.936021 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:22:45.675308 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:22:45.675275 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:22:46.442138 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:22:46.442099 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:22:46.542575 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:22:46.542544 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:22:49.277314 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:22:49.277279 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:22:49.377559 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:22:49.377520 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:22:50.038504 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:22:50.038469 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:22:50.132719 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:22:50.132693 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:22:50.365226 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:22:50.365182 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:22:50.461717 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:22:50.461680 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:22:53.511778 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:22:53.511743 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:22:53.904659 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:22:53.904626 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:22:55.882781 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:22:55.882682 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:22:55.938651 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:22:55.938612 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:22:59.424893 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:22:59.424858 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:22:59.525218 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:22:59.525181 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:01.113451 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:01.113418 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:03.160865 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:03.160819 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:03.261401 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:03.261367 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:08.698853 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:08.698799 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:08.799533 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:08.799501 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:10.579647 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:10.579610 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-59c77684cc-7jzgf"] Jun 15 09:23:10.581947 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:10.581927 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-7jzgf" Jun 15 09:23:10.584507 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:10.584487 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 15 09:23:10.584609 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:10.584514 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-r4d6t\"" Jun 15 09:23:10.584609 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:10.584487 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 15 09:23:10.591813 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:10.591789 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-7jzgf"] Jun 15 09:23:10.603174 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:10.603152 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ea17d6f1-e45f-4c7c-bc92-e7de4ba9b3c9-bound-sa-token\") pod \"cert-manager-59c77684cc-7jzgf\" (UID: \"ea17d6f1-e45f-4c7c-bc92-e7de4ba9b3c9\") " pod="cert-manager/cert-manager-59c77684cc-7jzgf" Jun 15 09:23:10.603284 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:10.603214 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmv4w\" (UniqueName: \"kubernetes.io/projected/ea17d6f1-e45f-4c7c-bc92-e7de4ba9b3c9-kube-api-access-pmv4w\") pod \"cert-manager-59c77684cc-7jzgf\" (UID: \"ea17d6f1-e45f-4c7c-bc92-e7de4ba9b3c9\") " pod="cert-manager/cert-manager-59c77684cc-7jzgf" Jun 15 09:23:10.610954 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:10.610930 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:10.704390 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:10.704362 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmv4w\" (UniqueName: \"kubernetes.io/projected/ea17d6f1-e45f-4c7c-bc92-e7de4ba9b3c9-kube-api-access-pmv4w\") pod \"cert-manager-59c77684cc-7jzgf\" (UID: \"ea17d6f1-e45f-4c7c-bc92-e7de4ba9b3c9\") " pod="cert-manager/cert-manager-59c77684cc-7jzgf" Jun 15 09:23:10.704565 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:10.704413 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ea17d6f1-e45f-4c7c-bc92-e7de4ba9b3c9-bound-sa-token\") pod \"cert-manager-59c77684cc-7jzgf\" (UID: \"ea17d6f1-e45f-4c7c-bc92-e7de4ba9b3c9\") " pod="cert-manager/cert-manager-59c77684cc-7jzgf" Jun 15 09:23:10.712679 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:10.712650 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ea17d6f1-e45f-4c7c-bc92-e7de4ba9b3c9-bound-sa-token\") pod \"cert-manager-59c77684cc-7jzgf\" (UID: \"ea17d6f1-e45f-4c7c-bc92-e7de4ba9b3c9\") " pod="cert-manager/cert-manager-59c77684cc-7jzgf" Jun 15 09:23:10.712969 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:10.712949 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmv4w\" (UniqueName: \"kubernetes.io/projected/ea17d6f1-e45f-4c7c-bc92-e7de4ba9b3c9-kube-api-access-pmv4w\") pod \"cert-manager-59c77684cc-7jzgf\" (UID: \"ea17d6f1-e45f-4c7c-bc92-e7de4ba9b3c9\") " pod="cert-manager/cert-manager-59c77684cc-7jzgf" Jun 15 09:23:10.891811 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:10.891717 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-7jzgf" Jun 15 09:23:11.010629 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:11.010601 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-7jzgf"] Jun 15 09:23:11.014624 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:23:11.014600 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea17d6f1_e45f_4c7c_bc92_e7de4ba9b3c9.slice/crio-e283c55acbf100e856ac35427e6a2c43ee4045f511a8c51d3da3cf44b13e0af6 WatchSource:0}: Error finding container e283c55acbf100e856ac35427e6a2c43ee4045f511a8c51d3da3cf44b13e0af6: Status 404 returned error can't find the container with id e283c55acbf100e856ac35427e6a2c43ee4045f511a8c51d3da3cf44b13e0af6 Jun 15 09:23:12.010164 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:12.010131 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-7jzgf" event={"ID":"ea17d6f1-e45f-4c7c-bc92-e7de4ba9b3c9","Type":"ContainerStarted","Data":"e283c55acbf100e856ac35427e6a2c43ee4045f511a8c51d3da3cf44b13e0af6"} Jun 15 09:23:15.023434 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:15.023394 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-7jzgf" event={"ID":"ea17d6f1-e45f-4c7c-bc92-e7de4ba9b3c9","Type":"ContainerStarted","Data":"a8b803962b48ab60d27f2baee2e7b7fb12be4b2873e55bc95d951d3e63310742"} Jun 15 09:23:15.039306 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:15.039260 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-59c77684cc-7jzgf" podStartSLOduration=1.55854052 podStartE2EDuration="5.03924758s" totalImagesPullingTime="3.48070706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:23:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:23:15.037900102 +0000 UTC m=+279.601322409" watchObservedRunningTime="2026-06-15 09:23:15.03924758 +0000 UTC m=+279.602669885" Jun 15 09:23:18.833284 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:18.833251 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:18.892183 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:18.892150 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-77b99946d4-8864g"] Jun 15 09:23:18.895275 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:18.895260 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-77b99946d4-8864g" Jun 15 09:23:18.898034 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:18.898014 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 15 09:23:18.898258 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:18.898245 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 15 09:23:18.899392 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:18.899374 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 15 09:23:18.899492 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:18.899415 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-5zvf2\"" Jun 15 09:23:18.899492 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:18.899445 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 15 09:23:18.899599 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:18.899516 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 15 09:23:18.905152 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:18.905131 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-77b99946d4-8864g"] Jun 15 09:23:18.909578 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:18.909555 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:18.964856 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:18.964811 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/28295990-b72d-473e-b481-1330cffbeaa7-metrics-cert\") pod \"lws-controller-manager-77b99946d4-8864g\" (UID: \"28295990-b72d-473e-b481-1330cffbeaa7\") " pod="openshift-lws-operator/lws-controller-manager-77b99946d4-8864g" Jun 15 09:23:18.964969 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:18.964865 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/28295990-b72d-473e-b481-1330cffbeaa7-manager-config\") pod \"lws-controller-manager-77b99946d4-8864g\" (UID: \"28295990-b72d-473e-b481-1330cffbeaa7\") " pod="openshift-lws-operator/lws-controller-manager-77b99946d4-8864g" Jun 15 09:23:18.964969 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:18.964927 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bbx5\" (UniqueName: \"kubernetes.io/projected/28295990-b72d-473e-b481-1330cffbeaa7-kube-api-access-5bbx5\") pod \"lws-controller-manager-77b99946d4-8864g\" (UID: \"28295990-b72d-473e-b481-1330cffbeaa7\") " pod="openshift-lws-operator/lws-controller-manager-77b99946d4-8864g" Jun 15 09:23:18.965073 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:18.965044 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28295990-b72d-473e-b481-1330cffbeaa7-cert\") pod \"lws-controller-manager-77b99946d4-8864g\" (UID: \"28295990-b72d-473e-b481-1330cffbeaa7\") " pod="openshift-lws-operator/lws-controller-manager-77b99946d4-8864g" Jun 15 09:23:19.065539 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:19.065507 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bbx5\" (UniqueName: \"kubernetes.io/projected/28295990-b72d-473e-b481-1330cffbeaa7-kube-api-access-5bbx5\") pod \"lws-controller-manager-77b99946d4-8864g\" (UID: \"28295990-b72d-473e-b481-1330cffbeaa7\") " pod="openshift-lws-operator/lws-controller-manager-77b99946d4-8864g" Jun 15 09:23:19.065645 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:19.065557 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28295990-b72d-473e-b481-1330cffbeaa7-cert\") pod \"lws-controller-manager-77b99946d4-8864g\" (UID: \"28295990-b72d-473e-b481-1330cffbeaa7\") " pod="openshift-lws-operator/lws-controller-manager-77b99946d4-8864g" Jun 15 09:23:19.065645 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:19.065582 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/28295990-b72d-473e-b481-1330cffbeaa7-metrics-cert\") pod \"lws-controller-manager-77b99946d4-8864g\" (UID: \"28295990-b72d-473e-b481-1330cffbeaa7\") " pod="openshift-lws-operator/lws-controller-manager-77b99946d4-8864g" Jun 15 09:23:19.065645 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:19.065603 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/28295990-b72d-473e-b481-1330cffbeaa7-manager-config\") pod \"lws-controller-manager-77b99946d4-8864g\" (UID: \"28295990-b72d-473e-b481-1330cffbeaa7\") " pod="openshift-lws-operator/lws-controller-manager-77b99946d4-8864g" Jun 15 09:23:19.066227 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:19.066201 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/28295990-b72d-473e-b481-1330cffbeaa7-manager-config\") pod \"lws-controller-manager-77b99946d4-8864g\" (UID: \"28295990-b72d-473e-b481-1330cffbeaa7\") " pod="openshift-lws-operator/lws-controller-manager-77b99946d4-8864g" Jun 15 09:23:19.067990 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:19.067965 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/28295990-b72d-473e-b481-1330cffbeaa7-metrics-cert\") pod \"lws-controller-manager-77b99946d4-8864g\" (UID: \"28295990-b72d-473e-b481-1330cffbeaa7\") " pod="openshift-lws-operator/lws-controller-manager-77b99946d4-8864g" Jun 15 09:23:19.068078 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:19.068022 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28295990-b72d-473e-b481-1330cffbeaa7-cert\") pod \"lws-controller-manager-77b99946d4-8864g\" (UID: \"28295990-b72d-473e-b481-1330cffbeaa7\") " pod="openshift-lws-operator/lws-controller-manager-77b99946d4-8864g" Jun 15 09:23:19.074921 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:19.074895 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bbx5\" (UniqueName: \"kubernetes.io/projected/28295990-b72d-473e-b481-1330cffbeaa7-kube-api-access-5bbx5\") pod \"lws-controller-manager-77b99946d4-8864g\" (UID: \"28295990-b72d-473e-b481-1330cffbeaa7\") " pod="openshift-lws-operator/lws-controller-manager-77b99946d4-8864g" Jun 15 09:23:19.204688 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:19.204621 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-77b99946d4-8864g" Jun 15 09:23:19.351210 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:19.351178 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-77b99946d4-8864g"] Jun 15 09:23:19.358925 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:23:19.358900 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28295990_b72d_473e_b481_1330cffbeaa7.slice/crio-0e2fcbefc0d390f5abc92f727b7b6bd02ba629aba74057f65c511db3e4f1727a WatchSource:0}: Error finding container 0e2fcbefc0d390f5abc92f727b7b6bd02ba629aba74057f65c511db3e4f1727a: Status 404 returned error can't find the container with id 0e2fcbefc0d390f5abc92f727b7b6bd02ba629aba74057f65c511db3e4f1727a Jun 15 09:23:20.041646 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:20.041617 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-77b99946d4-8864g" event={"ID":"28295990-b72d-473e-b481-1330cffbeaa7","Type":"ContainerStarted","Data":"0e2fcbefc0d390f5abc92f727b7b6bd02ba629aba74057f65c511db3e4f1727a"} Jun 15 09:23:23.052655 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:23.052621 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-77b99946d4-8864g" event={"ID":"28295990-b72d-473e-b481-1330cffbeaa7","Type":"ContainerStarted","Data":"fe1547a18c47dc0961b2049cc5a0a9d165aad29d81f6804faadce2b825c2915b"} Jun 15 09:23:23.053132 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:23.052693 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-77b99946d4-8864g" Jun 15 09:23:23.070485 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:23.070438 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-77b99946d4-8864g" podStartSLOduration=2.337423944 podStartE2EDuration="5.07042242s" totalImagesPullingTime="2.732998476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:23:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:23:23.069917285 +0000 UTC m=+287.633339605" watchObservedRunningTime="2026-06-15 09:23:23.07042242 +0000 UTC m=+287.633844774" Jun 15 09:23:27.114302 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:27.114270 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:27.208049 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:27.208016 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:27.715821 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:27.715783 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:27.909926 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:27.909895 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:28.022383 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.022295 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:28.063557 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.063523 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb"] Jun 15 09:23:28.066716 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.066697 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" Jun 15 09:23:28.071359 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.071341 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 15 09:23:28.071483 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.071390 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 15 09:23:28.071483 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.071445 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 15 09:23:28.071752 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.071734 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qvmw6\"" Jun 15 09:23:28.074194 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.074178 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 15 09:23:28.091393 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.091370 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb"] Jun 15 09:23:28.132219 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.132188 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/aafe0dfe-75d7-42b6-979c-a466e4edf40b-manifests\") pod \"opendatahub-operator-controller-manager-5659c68bbf-jjfvb\" (UID: \"aafe0dfe-75d7-42b6-979c-a466e4edf40b\") " pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" Jun 15 09:23:28.132607 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.132243 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aafe0dfe-75d7-42b6-979c-a466e4edf40b-tmp\") pod \"opendatahub-operator-controller-manager-5659c68bbf-jjfvb\" (UID: \"aafe0dfe-75d7-42b6-979c-a466e4edf40b\") " pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" Jun 15 09:23:28.132607 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.132318 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/aafe0dfe-75d7-42b6-979c-a466e4edf40b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5659c68bbf-jjfvb\" (UID: \"aafe0dfe-75d7-42b6-979c-a466e4edf40b\") " pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" Jun 15 09:23:28.132607 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.132371 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27hnt\" (UniqueName: \"kubernetes.io/projected/aafe0dfe-75d7-42b6-979c-a466e4edf40b-kube-api-access-27hnt\") pod \"opendatahub-operator-controller-manager-5659c68bbf-jjfvb\" (UID: \"aafe0dfe-75d7-42b6-979c-a466e4edf40b\") " pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" Jun 15 09:23:28.132607 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.132416 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aafe0dfe-75d7-42b6-979c-a466e4edf40b-webhook-cert\") pod \"opendatahub-operator-controller-manager-5659c68bbf-jjfvb\" (UID: \"aafe0dfe-75d7-42b6-979c-a466e4edf40b\") " pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" Jun 15 09:23:28.233075 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.233032 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aafe0dfe-75d7-42b6-979c-a466e4edf40b-tmp\") pod \"opendatahub-operator-controller-manager-5659c68bbf-jjfvb\" (UID: \"aafe0dfe-75d7-42b6-979c-a466e4edf40b\") " pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" Jun 15 09:23:28.233276 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.233143 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/aafe0dfe-75d7-42b6-979c-a466e4edf40b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5659c68bbf-jjfvb\" (UID: \"aafe0dfe-75d7-42b6-979c-a466e4edf40b\") " pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" Jun 15 09:23:28.233276 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.233168 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27hnt\" (UniqueName: \"kubernetes.io/projected/aafe0dfe-75d7-42b6-979c-a466e4edf40b-kube-api-access-27hnt\") pod \"opendatahub-operator-controller-manager-5659c68bbf-jjfvb\" (UID: \"aafe0dfe-75d7-42b6-979c-a466e4edf40b\") " pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" Jun 15 09:23:28.233276 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.233188 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aafe0dfe-75d7-42b6-979c-a466e4edf40b-webhook-cert\") pod \"opendatahub-operator-controller-manager-5659c68bbf-jjfvb\" (UID: \"aafe0dfe-75d7-42b6-979c-a466e4edf40b\") " pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" Jun 15 09:23:28.233276 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.233210 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/aafe0dfe-75d7-42b6-979c-a466e4edf40b-manifests\") pod \"opendatahub-operator-controller-manager-5659c68bbf-jjfvb\" (UID: \"aafe0dfe-75d7-42b6-979c-a466e4edf40b\") " pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" Jun 15 09:23:28.233616 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.233572 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/aafe0dfe-75d7-42b6-979c-a466e4edf40b-manifests\") pod \"opendatahub-operator-controller-manager-5659c68bbf-jjfvb\" (UID: \"aafe0dfe-75d7-42b6-979c-a466e4edf40b\") " pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" Jun 15 09:23:28.235380 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.235318 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:28.235679 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.235656 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aafe0dfe-75d7-42b6-979c-a466e4edf40b-tmp\") pod \"opendatahub-operator-controller-manager-5659c68bbf-jjfvb\" (UID: \"aafe0dfe-75d7-42b6-979c-a466e4edf40b\") " pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" Jun 15 09:23:28.235748 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.235735 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/aafe0dfe-75d7-42b6-979c-a466e4edf40b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5659c68bbf-jjfvb\" (UID: \"aafe0dfe-75d7-42b6-979c-a466e4edf40b\") " pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" Jun 15 09:23:28.235852 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.235819 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aafe0dfe-75d7-42b6-979c-a466e4edf40b-webhook-cert\") pod \"opendatahub-operator-controller-manager-5659c68bbf-jjfvb\" (UID: \"aafe0dfe-75d7-42b6-979c-a466e4edf40b\") " pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" Jun 15 09:23:28.241370 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.241346 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27hnt\" (UniqueName: \"kubernetes.io/projected/aafe0dfe-75d7-42b6-979c-a466e4edf40b-kube-api-access-27hnt\") pod \"opendatahub-operator-controller-manager-5659c68bbf-jjfvb\" (UID: \"aafe0dfe-75d7-42b6-979c-a466e4edf40b\") " pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" Jun 15 09:23:28.346600 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.346558 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:28.377228 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.377197 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" Jun 15 09:23:28.449570 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.449525 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:28.515221 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.515195 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb"] Jun 15 09:23:28.518288 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:23:28.518255 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaafe0dfe_75d7_42b6_979c_a466e4edf40b.slice/crio-9245ce8f5dfd0e9d5d5766ad2636258015e9cfc91a62f732bbcf8e578c7ad087 WatchSource:0}: Error finding container 9245ce8f5dfd0e9d5d5766ad2636258015e9cfc91a62f732bbcf8e578c7ad087: Status 404 returned error can't find the container with id 9245ce8f5dfd0e9d5d5766ad2636258015e9cfc91a62f732bbcf8e578c7ad087 Jun 15 09:23:28.518437 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.518294 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:28.574259 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.574223 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:28.675386 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.675296 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:28.775447 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:28.775399 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:29.076037 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:29.075995 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" event={"ID":"aafe0dfe-75d7-42b6-979c-a466e4edf40b","Type":"ContainerStarted","Data":"9245ce8f5dfd0e9d5d5766ad2636258015e9cfc91a62f732bbcf8e578c7ad087"} Jun 15 09:23:32.089563 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:32.089530 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" event={"ID":"aafe0dfe-75d7-42b6-979c-a466e4edf40b","Type":"ContainerStarted","Data":"6062d2cf28dc1d99235b16adbe041cee8df0f5cf332eecb28dcf8beef4f0cc59"} Jun 15 09:23:33.095121 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:33.095086 2562 generic.go:356] "Generic (PLEG): container finished" podID="aafe0dfe-75d7-42b6-979c-a466e4edf40b" containerID="6062d2cf28dc1d99235b16adbe041cee8df0f5cf332eecb28dcf8beef4f0cc59" exitCode=0 Jun 15 09:23:33.095527 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:33.095133 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" event={"ID":"aafe0dfe-75d7-42b6-979c-a466e4edf40b","Type":"ContainerDied","Data":"6062d2cf28dc1d99235b16adbe041cee8df0f5cf332eecb28dcf8beef4f0cc59"} Jun 15 09:23:34.057982 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:34.057945 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-77b99946d4-8864g" Jun 15 09:23:34.101987 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:34.101955 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" event={"ID":"aafe0dfe-75d7-42b6-979c-a466e4edf40b","Type":"ContainerStarted","Data":"333ecc0f843ad2bac06605238ef63804bb89bb213a62b63a7401e0f8be18d7dc"} Jun 15 09:23:34.102448 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:34.102070 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" Jun 15 09:23:34.127606 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:34.127549 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" podStartSLOduration=1.9567028579999999 podStartE2EDuration="6.1275324s" totalImagesPullingTime="3.242173477s" totalInitContainerRuntime="928.656065ms" isStatefulPod=false podCreationTimestamp="2026-06-15 09:23:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:23:34.120052244 +0000 UTC m=+298.683474548" watchObservedRunningTime="2026-06-15 09:23:34.1275324 +0000 UTC m=+298.690954704" Jun 15 09:23:35.869200 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:35.869167 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pxdxj_f344540f-2677-49dd-8a0f-106be00f414d/ovn-acl-logging/0.log" Jun 15 09:23:35.869688 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:35.869669 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pxdxj_f344540f-2677-49dd-8a0f-106be00f414d/ovn-acl-logging/0.log" Jun 15 09:23:35.877092 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:35.877071 2562 kubelet.go:1648] "Image garbage collection succeeded" Jun 15 09:23:39.323903 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:39.323869 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:40.335915 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:40.335880 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:45.107951 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:45.107908 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5659c68bbf-jjfvb" Jun 15 09:23:46.824520 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:46.824478 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:46.935629 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:46.935591 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:47.111974 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.111881 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:47.204482 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.204451 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:47.331509 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.331474 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:47.457779 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.457698 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6bcbb8b488-cr726"] Jun 15 09:23:47.484772 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.484736 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:47.494120 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.494096 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bcbb8b488-cr726"] Jun 15 09:23:47.494244 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.494201 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bcbb8b488-cr726" Jun 15 09:23:47.496944 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.496924 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 15 09:23:47.497910 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.497896 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 15 09:23:47.498243 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.498220 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 15 09:23:47.498422 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.498226 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-gsn9p\"" Jun 15 09:23:47.498422 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.498243 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 15 09:23:47.581621 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.581595 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a85a02-7443-41b3-a429-ef011dc7f3c2-tls-certs\") pod \"kube-auth-proxy-6bcbb8b488-cr726\" (UID: \"28a85a02-7443-41b3-a429-ef011dc7f3c2\") " pod="openshift-ingress/kube-auth-proxy-6bcbb8b488-cr726" Jun 15 09:23:47.581794 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.581641 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28a85a02-7443-41b3-a429-ef011dc7f3c2-tmp\") pod \"kube-auth-proxy-6bcbb8b488-cr726\" (UID: \"28a85a02-7443-41b3-a429-ef011dc7f3c2\") " pod="openshift-ingress/kube-auth-proxy-6bcbb8b488-cr726" Jun 15 09:23:47.581794 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.581668 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvwhb\" (UniqueName: \"kubernetes.io/projected/28a85a02-7443-41b3-a429-ef011dc7f3c2-kube-api-access-nvwhb\") pod \"kube-auth-proxy-6bcbb8b488-cr726\" (UID: \"28a85a02-7443-41b3-a429-ef011dc7f3c2\") " pod="openshift-ingress/kube-auth-proxy-6bcbb8b488-cr726" Jun 15 09:23:47.683102 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.683074 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvwhb\" (UniqueName: \"kubernetes.io/projected/28a85a02-7443-41b3-a429-ef011dc7f3c2-kube-api-access-nvwhb\") pod \"kube-auth-proxy-6bcbb8b488-cr726\" (UID: \"28a85a02-7443-41b3-a429-ef011dc7f3c2\") " pod="openshift-ingress/kube-auth-proxy-6bcbb8b488-cr726" Jun 15 09:23:47.683314 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.683115 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a85a02-7443-41b3-a429-ef011dc7f3c2-tls-certs\") pod \"kube-auth-proxy-6bcbb8b488-cr726\" (UID: \"28a85a02-7443-41b3-a429-ef011dc7f3c2\") " pod="openshift-ingress/kube-auth-proxy-6bcbb8b488-cr726" Jun 15 09:23:47.683314 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.683155 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28a85a02-7443-41b3-a429-ef011dc7f3c2-tmp\") pod \"kube-auth-proxy-6bcbb8b488-cr726\" (UID: \"28a85a02-7443-41b3-a429-ef011dc7f3c2\") " pod="openshift-ingress/kube-auth-proxy-6bcbb8b488-cr726" Jun 15 09:23:47.685460 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.685434 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28a85a02-7443-41b3-a429-ef011dc7f3c2-tmp\") pod \"kube-auth-proxy-6bcbb8b488-cr726\" (UID: \"28a85a02-7443-41b3-a429-ef011dc7f3c2\") " pod="openshift-ingress/kube-auth-proxy-6bcbb8b488-cr726" Jun 15 09:23:47.685656 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.685638 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a85a02-7443-41b3-a429-ef011dc7f3c2-tls-certs\") pod \"kube-auth-proxy-6bcbb8b488-cr726\" (UID: \"28a85a02-7443-41b3-a429-ef011dc7f3c2\") " pod="openshift-ingress/kube-auth-proxy-6bcbb8b488-cr726" Jun 15 09:23:47.699249 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.699227 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvwhb\" (UniqueName: \"kubernetes.io/projected/28a85a02-7443-41b3-a429-ef011dc7f3c2-kube-api-access-nvwhb\") pod \"kube-auth-proxy-6bcbb8b488-cr726\" (UID: \"28a85a02-7443-41b3-a429-ef011dc7f3c2\") " pod="openshift-ingress/kube-auth-proxy-6bcbb8b488-cr726" Jun 15 09:23:47.717857 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.717767 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:47.803547 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.803522 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bcbb8b488-cr726" Jun 15 09:23:47.824038 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.824008 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:47.904485 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.904448 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:47.968452 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.968429 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bcbb8b488-cr726"] Jun 15 09:23:47.970868 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:23:47.970823 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28a85a02_7443_41b3_a429_ef011dc7f3c2.slice/crio-92cc8187161b197f351221c48381e01b566d2b0e1386cad09ec02f21b4a2c264 WatchSource:0}: Error finding container 92cc8187161b197f351221c48381e01b566d2b0e1386cad09ec02f21b4a2c264: Status 404 returned error can't find the container with id 92cc8187161b197f351221c48381e01b566d2b0e1386cad09ec02f21b4a2c264 Jun 15 09:23:47.972750 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:47.972734 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 15 09:23:48.151194 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:48.151159 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bcbb8b488-cr726" event={"ID":"28a85a02-7443-41b3-a429-ef011dc7f3c2","Type":"ContainerStarted","Data":"92cc8187161b197f351221c48381e01b566d2b0e1386cad09ec02f21b4a2c264"} Jun 15 09:23:48.323537 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:48.323504 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:48.453107 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:48.453074 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:52.167037 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:52.167001 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bcbb8b488-cr726" event={"ID":"28a85a02-7443-41b3-a429-ef011dc7f3c2","Type":"ContainerStarted","Data":"84d4b12c677417cb30e06799b1226b39c12dbd3f11c20a7a125f2e368d23cb58"} Jun 15 09:23:52.183432 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:52.183386 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6bcbb8b488-cr726" podStartSLOduration=1.903779445 podStartE2EDuration="5.183347107s" totalImagesPullingTime="3.279567662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:23:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:23:52.182556429 +0000 UTC m=+316.745978736" watchObservedRunningTime="2026-06-15 09:23:52.183347107 +0000 UTC m=+316.746769411" Jun 15 09:23:58.312145 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:58.312107 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:58.394192 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:58.394157 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:58.480942 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:58.480910 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:58.582467 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:58.582436 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:59.184587 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:59.184554 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:59.316354 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:59.316322 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:59.658441 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:59.658413 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:23:59.825500 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:23:59.825470 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:24:00.078231 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:24:00.078195 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:24:01.137450 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:24:01.137419 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:24:01.256359 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:24:01.256326 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:24:01.692399 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:24:01.692362 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:24:01.854142 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:24:01.854109 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:24:03.393811 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:24:03.393781 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:24:18.387690 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:24:18.387650 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:24:18.474065 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:24:18.474019 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:24:18.907528 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:24:18.907493 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:24:22.175487 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:24:22.175455 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:24:22.234771 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:24:22.234740 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:24:22.463380 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:24:22.463311 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:24:22.562360 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:24:22.562328 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:24:22.870946 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:24:22.870915 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:24:23.435690 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:24:23.435654 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:24:48.318515 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:24:48.318439 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:24:48.377986 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:24:48.377945 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:24:48.491385 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:24:48.491346 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:24:49.237438 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:24:49.237401 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:24:49.439943 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:24:49.439907 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:06.606563 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:06.606520 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:06.662739 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:06.662701 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:08.782494 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:08.782456 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:08.884222 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:08.884188 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:17.059317 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:17.059283 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:20.923020 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:20.922952 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:22.252018 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:22.251990 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:22.660519 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:22.660485 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:23.702447 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:23.702414 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:25.266667 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:25.266635 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:25.362552 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:25.362525 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:27.469487 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:27.469448 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:28.894912 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:28.894872 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-nzss9"] Jun 15 09:25:28.898262 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:28.898241 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nzss9" Jun 15 09:25:28.900875 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:28.900851 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 15 09:25:28.901058 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:28.901044 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 15 09:25:28.901300 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:28.901287 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hx2b7\"" Jun 15 09:25:28.901970 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:28.901955 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 15 09:25:28.902029 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:28.901958 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 15 09:25:28.904896 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:28.904870 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:28.913205 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:28.913182 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-nzss9"] Jun 15 09:25:29.048766 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:29.048729 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97pxh\" (UniqueName: \"kubernetes.io/projected/9fa3a695-3004-4447-bc80-c3a5b102cf71-kube-api-access-97pxh\") pod \"kuadrant-console-plugin-66b5774985-nzss9\" (UID: \"9fa3a695-3004-4447-bc80-c3a5b102cf71\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nzss9" Jun 15 09:25:29.048766 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:29.048772 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9fa3a695-3004-4447-bc80-c3a5b102cf71-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nzss9\" (UID: \"9fa3a695-3004-4447-bc80-c3a5b102cf71\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nzss9" Jun 15 09:25:29.049064 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:29.048916 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9fa3a695-3004-4447-bc80-c3a5b102cf71-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-nzss9\" (UID: \"9fa3a695-3004-4447-bc80-c3a5b102cf71\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nzss9" Jun 15 09:25:29.150362 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:29.150273 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9fa3a695-3004-4447-bc80-c3a5b102cf71-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-nzss9\" (UID: \"9fa3a695-3004-4447-bc80-c3a5b102cf71\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nzss9" Jun 15 09:25:29.150362 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:29.150322 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97pxh\" (UniqueName: \"kubernetes.io/projected/9fa3a695-3004-4447-bc80-c3a5b102cf71-kube-api-access-97pxh\") pod \"kuadrant-console-plugin-66b5774985-nzss9\" (UID: \"9fa3a695-3004-4447-bc80-c3a5b102cf71\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nzss9" Jun 15 09:25:29.150566 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:29.150437 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9fa3a695-3004-4447-bc80-c3a5b102cf71-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nzss9\" (UID: \"9fa3a695-3004-4447-bc80-c3a5b102cf71\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nzss9" Jun 15 09:25:29.150566 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:25:29.150537 2562 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jun 15 09:25:29.150667 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:25:29.150613 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fa3a695-3004-4447-bc80-c3a5b102cf71-plugin-serving-cert podName:9fa3a695-3004-4447-bc80-c3a5b102cf71 nodeName:}" failed. No retries permitted until 2026-06-15 09:25:29.650591072 +0000 UTC m=+414.214013366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/9fa3a695-3004-4447-bc80-c3a5b102cf71-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-nzss9" (UID: "9fa3a695-3004-4447-bc80-c3a5b102cf71") : secret "plugin-serving-cert" not found Jun 15 09:25:29.150947 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:29.150926 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9fa3a695-3004-4447-bc80-c3a5b102cf71-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-nzss9\" (UID: \"9fa3a695-3004-4447-bc80-c3a5b102cf71\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nzss9" Jun 15 09:25:29.158533 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:29.158501 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97pxh\" (UniqueName: \"kubernetes.io/projected/9fa3a695-3004-4447-bc80-c3a5b102cf71-kube-api-access-97pxh\") pod \"kuadrant-console-plugin-66b5774985-nzss9\" (UID: \"9fa3a695-3004-4447-bc80-c3a5b102cf71\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nzss9" Jun 15 09:25:29.250307 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:29.250263 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:29.495634 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:29.495545 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:29.596141 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:29.596108 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:29.654750 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:29.654709 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9fa3a695-3004-4447-bc80-c3a5b102cf71-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nzss9\" (UID: \"9fa3a695-3004-4447-bc80-c3a5b102cf71\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nzss9" Jun 15 09:25:29.657150 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:29.657123 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9fa3a695-3004-4447-bc80-c3a5b102cf71-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nzss9\" (UID: \"9fa3a695-3004-4447-bc80-c3a5b102cf71\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nzss9" Jun 15 09:25:29.697288 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:29.697260 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:29.809098 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:29.809014 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nzss9" Jun 15 09:25:29.941557 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:29.941528 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-nzss9"] Jun 15 09:25:29.944659 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:25:29.944635 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9fa3a695_3004_4447_bc80_c3a5b102cf71.slice/crio-58330b19cae7f827bc40f47d59b55e2ce1ba2026eba0bd37c01c71ff39d6da34 WatchSource:0}: Error finding container 58330b19cae7f827bc40f47d59b55e2ce1ba2026eba0bd37c01c71ff39d6da34: Status 404 returned error can't find the container with id 58330b19cae7f827bc40f47d59b55e2ce1ba2026eba0bd37c01c71ff39d6da34 Jun 15 09:25:30.521145 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:30.521110 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nzss9" event={"ID":"9fa3a695-3004-4447-bc80-c3a5b102cf71","Type":"ContainerStarted","Data":"58330b19cae7f827bc40f47d59b55e2ce1ba2026eba0bd37c01c71ff39d6da34"} Jun 15 09:25:31.867320 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:31.867282 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:31.967621 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:31.967577 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:38.171016 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:38.170980 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cq6fh"] Jun 15 09:25:38.191487 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:38.191459 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cq6fh"] Jun 15 09:25:38.191673 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:38.191587 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cq6fh" Jun 15 09:25:38.194347 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:38.194322 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-r5d74\"" Jun 15 09:25:38.330847 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:38.330794 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8cjs\" (UniqueName: \"kubernetes.io/projected/ac20a91d-48db-43fb-ace9-e975bf2dbb4a-kube-api-access-q8cjs\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cq6fh\" (UID: \"ac20a91d-48db-43fb-ace9-e975bf2dbb4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cq6fh" Jun 15 09:25:38.331034 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:38.330929 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac20a91d-48db-43fb-ace9-e975bf2dbb4a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cq6fh\" (UID: \"ac20a91d-48db-43fb-ace9-e975bf2dbb4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cq6fh" Jun 15 09:25:38.432312 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:38.432217 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8cjs\" (UniqueName: \"kubernetes.io/projected/ac20a91d-48db-43fb-ace9-e975bf2dbb4a-kube-api-access-q8cjs\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cq6fh\" (UID: \"ac20a91d-48db-43fb-ace9-e975bf2dbb4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cq6fh" Jun 15 09:25:38.432488 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:38.432384 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac20a91d-48db-43fb-ace9-e975bf2dbb4a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cq6fh\" (UID: \"ac20a91d-48db-43fb-ace9-e975bf2dbb4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cq6fh" Jun 15 09:25:38.432752 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:38.432734 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac20a91d-48db-43fb-ace9-e975bf2dbb4a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cq6fh\" (UID: \"ac20a91d-48db-43fb-ace9-e975bf2dbb4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cq6fh" Jun 15 09:25:38.443503 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:38.443463 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8cjs\" (UniqueName: \"kubernetes.io/projected/ac20a91d-48db-43fb-ace9-e975bf2dbb4a-kube-api-access-q8cjs\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cq6fh\" (UID: \"ac20a91d-48db-43fb-ace9-e975bf2dbb4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cq6fh" Jun 15 09:25:38.502095 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:38.502062 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cq6fh"] Jun 15 09:25:38.502390 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:38.502374 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cq6fh" Jun 15 09:25:38.756990 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:38.756903 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cq6fh"] Jun 15 09:25:38.772561 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:38.772525 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cq6fh"] Jun 15 09:25:38.816402 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:38.816313 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s5n7"] Jun 15 09:25:38.829117 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:38.829086 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s5n7" Jun 15 09:25:38.832966 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:38.832936 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s5n7"] Jun 15 09:25:38.938080 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:38.938045 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fps8r\" (UniqueName: \"kubernetes.io/projected/89ca778b-35f9-41e3-bdf1-f51b3d29f220-kube-api-access-fps8r\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7s5n7\" (UID: \"89ca778b-35f9-41e3-bdf1-f51b3d29f220\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s5n7" Jun 15 09:25:38.938243 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:38.938086 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89ca778b-35f9-41e3-bdf1-f51b3d29f220-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7s5n7\" (UID: \"89ca778b-35f9-41e3-bdf1-f51b3d29f220\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s5n7" Jun 15 09:25:39.039148 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:39.039058 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fps8r\" (UniqueName: \"kubernetes.io/projected/89ca778b-35f9-41e3-bdf1-f51b3d29f220-kube-api-access-fps8r\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7s5n7\" (UID: \"89ca778b-35f9-41e3-bdf1-f51b3d29f220\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s5n7" Jun 15 09:25:39.039148 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:39.039109 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89ca778b-35f9-41e3-bdf1-f51b3d29f220-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7s5n7\" (UID: \"89ca778b-35f9-41e3-bdf1-f51b3d29f220\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s5n7" Jun 15 09:25:39.039528 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:39.039499 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89ca778b-35f9-41e3-bdf1-f51b3d29f220-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7s5n7\" (UID: \"89ca778b-35f9-41e3-bdf1-f51b3d29f220\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s5n7" Jun 15 09:25:39.047721 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:39.047684 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fps8r\" (UniqueName: \"kubernetes.io/projected/89ca778b-35f9-41e3-bdf1-f51b3d29f220-kube-api-access-fps8r\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7s5n7\" (UID: \"89ca778b-35f9-41e3-bdf1-f51b3d29f220\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s5n7" Jun 15 09:25:39.144509 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:39.144472 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s5n7" Jun 15 09:25:40.245547 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:40.245503 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:40.329491 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:40.329449 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:41.871571 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:41.871535 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:41.974250 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:41.974198 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:42.459044 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:42.459011 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:42.562817 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:42.562774 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:46.264605 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:46.264570 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:46.364960 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:46.364924 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:50.262579 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:50.262539 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:50.363660 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:50.363622 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:50.779605 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:50.779580 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s5n7"] Jun 15 09:25:50.781207 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:25:50.781182 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89ca778b_35f9_41e3_bdf1_f51b3d29f220.slice/crio-00b6280a93a871d92b4b971350c176233cc11036e6d21cca26aa9a9ed23b5321 WatchSource:0}: Error finding container 00b6280a93a871d92b4b971350c176233cc11036e6d21cca26aa9a9ed23b5321: Status 404 returned error can't find the container with id 00b6280a93a871d92b4b971350c176233cc11036e6d21cca26aa9a9ed23b5321 Jun 15 09:25:51.059997 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:51.059918 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s5n7"] Jun 15 09:25:51.613451 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:51.613421 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nzss9" event={"ID":"9fa3a695-3004-4447-bc80-c3a5b102cf71","Type":"ContainerStarted","Data":"cb5319f13dabadd73f084c2bce647fab1d25682176a8508c2ac40f88ce41c2b5"} Jun 15 09:25:51.614859 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:51.614820 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s5n7" event={"ID":"89ca778b-35f9-41e3-bdf1-f51b3d29f220","Type":"ContainerStarted","Data":"00b6280a93a871d92b4b971350c176233cc11036e6d21cca26aa9a9ed23b5321"} Jun 15 09:25:51.629125 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:51.629063 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nzss9" podStartSLOduration=2.846607548 podStartE2EDuration="23.629042812s" totalImagesPullingTime="20.782435264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:25:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:25:51.628474195 +0000 UTC m=+436.191896493" watchObservedRunningTime="2026-06-15 09:25:51.629042812 +0000 UTC m=+436.192465118" Jun 15 09:25:53.184064 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:25:53.184017 2562 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-cq6fh_kuadrant-system_ac20a91d-48db-43fb-ace9-e975bf2dbb4a_0(07706d7a8324a12a02229a20541ba6bc1a64f53c81f7cb11c5a608c3162bd228): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-cq6fh to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jun 15 09:25:53.184397 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:25:53.184086 2562 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-cq6fh_kuadrant-system_ac20a91d-48db-43fb-ace9-e975bf2dbb4a_0(07706d7a8324a12a02229a20541ba6bc1a64f53c81f7cb11c5a608c3162bd228): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-cq6fh to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cq6fh" Jun 15 09:25:53.623064 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:53.623028 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cq6fh" Jun 15 09:25:53.625400 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:25:53.625359 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-cq6fh\" is forbidden: User \"system:node:ip-10-0-129-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-143.ec2.internal' and this object" podUID="ac20a91d-48db-43fb-ace9-e975bf2dbb4a" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cq6fh" Jun 15 09:25:53.632105 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:53.632082 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cq6fh" Jun 15 09:25:53.634405 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:25:53.634372 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-cq6fh\" is forbidden: User \"system:node:ip-10-0-129-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-143.ec2.internal' and this object" podUID="ac20a91d-48db-43fb-ace9-e975bf2dbb4a" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cq6fh" Jun 15 09:25:53.663821 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:53.663791 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac20a91d-48db-43fb-ace9-e975bf2dbb4a-extensions-socket-volume\") pod \"ac20a91d-48db-43fb-ace9-e975bf2dbb4a\" (UID: \"ac20a91d-48db-43fb-ace9-e975bf2dbb4a\") " Jun 15 09:25:53.664037 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:53.663867 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8cjs\" (UniqueName: \"kubernetes.io/projected/ac20a91d-48db-43fb-ace9-e975bf2dbb4a-kube-api-access-q8cjs\") pod \"ac20a91d-48db-43fb-ace9-e975bf2dbb4a\" (UID: \"ac20a91d-48db-43fb-ace9-e975bf2dbb4a\") " Jun 15 09:25:53.664175 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:53.664085 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac20a91d-48db-43fb-ace9-e975bf2dbb4a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ac20a91d-48db-43fb-ace9-e975bf2dbb4a" (UID: "ac20a91d-48db-43fb-ace9-e975bf2dbb4a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 09:25:53.664175 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:53.664149 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac20a91d-48db-43fb-ace9-e975bf2dbb4a-extensions-socket-volume\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:25:53.666163 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:53.666139 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac20a91d-48db-43fb-ace9-e975bf2dbb4a-kube-api-access-q8cjs" (OuterVolumeSpecName: "kube-api-access-q8cjs") pod "ac20a91d-48db-43fb-ace9-e975bf2dbb4a" (UID: "ac20a91d-48db-43fb-ace9-e975bf2dbb4a"). InnerVolumeSpecName "kube-api-access-q8cjs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 09:25:53.764705 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:53.764672 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8cjs\" (UniqueName: \"kubernetes.io/projected/ac20a91d-48db-43fb-ace9-e975bf2dbb4a-kube-api-access-q8cjs\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:25:53.935974 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:53.935896 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac20a91d-48db-43fb-ace9-e975bf2dbb4a" path="/var/lib/kubelet/pods/ac20a91d-48db-43fb-ace9-e975bf2dbb4a/volumes" Jun 15 09:25:54.063964 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:54.063916 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:54.170478 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:54.170424 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:25:54.627804 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:54.627720 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cq6fh" Jun 15 09:25:54.630083 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:25:54.630055 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-cq6fh\" is forbidden: User \"system:node:ip-10-0-129-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-143.ec2.internal' and this object" podUID="ac20a91d-48db-43fb-ace9-e975bf2dbb4a" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cq6fh" Jun 15 09:25:54.632047 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:25:54.632020 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-cq6fh\" is forbidden: User \"system:node:ip-10-0-129-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-143.ec2.internal' and this object" podUID="ac20a91d-48db-43fb-ace9-e975bf2dbb4a" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cq6fh" Jun 15 09:25:55.632623 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:55.632580 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s5n7" event={"ID":"89ca778b-35f9-41e3-bdf1-f51b3d29f220","Type":"ContainerStarted","Data":"f206bf189ec8fed5d361d2a0839a421784914f6c063c6bb366744813f2567542"} Jun 15 09:25:55.633054 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:55.632636 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s5n7" podUID="89ca778b-35f9-41e3-bdf1-f51b3d29f220" containerName="manager" containerID="cri-o://f206bf189ec8fed5d361d2a0839a421784914f6c063c6bb366744813f2567542" gracePeriod=10 Jun 15 09:25:55.633054 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:55.632722 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s5n7" Jun 15 09:25:55.652422 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:25:55.652388 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-cq6fh\" is forbidden: User \"system:node:ip-10-0-129-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-143.ec2.internal' and this object" podUID="ac20a91d-48db-43fb-ace9-e975bf2dbb4a" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cq6fh" Jun 15 09:25:55.652574 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:55.652457 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s5n7" podStartSLOduration=13.304744573 podStartE2EDuration="17.652441701s" totalImagesPullingTime="4.347697128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:25:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:25:55.650469446 +0000 UTC m=+440.213891785" watchObservedRunningTime="2026-06-15 09:25:55.652441701 +0000 UTC m=+440.215864006" Jun 15 09:25:55.933547 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:25:55.933455 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-cq6fh\" is forbidden: User \"system:node:ip-10-0-129-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-143.ec2.internal' and this object" podUID="ac20a91d-48db-43fb-ace9-e975bf2dbb4a" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cq6fh" Jun 15 09:25:56.382563 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:56.382536 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s5n7" Jun 15 09:25:56.489852 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:56.489793 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fps8r\" (UniqueName: \"kubernetes.io/projected/89ca778b-35f9-41e3-bdf1-f51b3d29f220-kube-api-access-fps8r\") pod \"89ca778b-35f9-41e3-bdf1-f51b3d29f220\" (UID: \"89ca778b-35f9-41e3-bdf1-f51b3d29f220\") " Jun 15 09:25:56.490023 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:56.489890 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89ca778b-35f9-41e3-bdf1-f51b3d29f220-extensions-socket-volume\") pod \"89ca778b-35f9-41e3-bdf1-f51b3d29f220\" (UID: \"89ca778b-35f9-41e3-bdf1-f51b3d29f220\") " Jun 15 09:25:56.490355 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:56.490318 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89ca778b-35f9-41e3-bdf1-f51b3d29f220-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "89ca778b-35f9-41e3-bdf1-f51b3d29f220" (UID: "89ca778b-35f9-41e3-bdf1-f51b3d29f220"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 09:25:56.492074 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:56.492050 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89ca778b-35f9-41e3-bdf1-f51b3d29f220-kube-api-access-fps8r" (OuterVolumeSpecName: "kube-api-access-fps8r") pod "89ca778b-35f9-41e3-bdf1-f51b3d29f220" (UID: "89ca778b-35f9-41e3-bdf1-f51b3d29f220"). InnerVolumeSpecName "kube-api-access-fps8r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 09:25:56.591122 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:56.591086 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fps8r\" (UniqueName: \"kubernetes.io/projected/89ca778b-35f9-41e3-bdf1-f51b3d29f220-kube-api-access-fps8r\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:25:56.591122 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:56.591116 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89ca778b-35f9-41e3-bdf1-f51b3d29f220-extensions-socket-volume\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:25:56.637945 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:56.637909 2562 generic.go:356] "Generic (PLEG): container finished" podID="89ca778b-35f9-41e3-bdf1-f51b3d29f220" containerID="f206bf189ec8fed5d361d2a0839a421784914f6c063c6bb366744813f2567542" exitCode=0 Jun 15 09:25:56.638308 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:56.637953 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s5n7" Jun 15 09:25:56.638308 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:56.637993 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s5n7" event={"ID":"89ca778b-35f9-41e3-bdf1-f51b3d29f220","Type":"ContainerDied","Data":"f206bf189ec8fed5d361d2a0839a421784914f6c063c6bb366744813f2567542"} Jun 15 09:25:56.638308 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:56.638043 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s5n7" event={"ID":"89ca778b-35f9-41e3-bdf1-f51b3d29f220","Type":"ContainerDied","Data":"00b6280a93a871d92b4b971350c176233cc11036e6d21cca26aa9a9ed23b5321"} Jun 15 09:25:56.638308 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:56.638066 2562 scope.go:117] "RemoveContainer" containerID="f206bf189ec8fed5d361d2a0839a421784914f6c063c6bb366744813f2567542" Jun 15 09:25:56.647202 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:56.647185 2562 scope.go:117] "RemoveContainer" containerID="f206bf189ec8fed5d361d2a0839a421784914f6c063c6bb366744813f2567542" Jun 15 09:25:56.647463 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:25:56.647446 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f206bf189ec8fed5d361d2a0839a421784914f6c063c6bb366744813f2567542\": container with ID starting with f206bf189ec8fed5d361d2a0839a421784914f6c063c6bb366744813f2567542 not found: ID does not exist" containerID="f206bf189ec8fed5d361d2a0839a421784914f6c063c6bb366744813f2567542" Jun 15 09:25:56.647520 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:56.647470 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f206bf189ec8fed5d361d2a0839a421784914f6c063c6bb366744813f2567542"} err="failed to get container status \"f206bf189ec8fed5d361d2a0839a421784914f6c063c6bb366744813f2567542\": rpc error: code = NotFound desc = could not find container \"f206bf189ec8fed5d361d2a0839a421784914f6c063c6bb366744813f2567542\": container with ID starting with f206bf189ec8fed5d361d2a0839a421784914f6c063c6bb366744813f2567542 not found: ID does not exist" Jun 15 09:25:56.658955 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:56.658930 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s5n7"] Jun 15 09:25:56.663383 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:56.663362 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7s5n7"] Jun 15 09:25:57.934438 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:25:57.934408 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89ca778b-35f9-41e3-bdf1-f51b3d29f220" path="/var/lib/kubelet/pods/89ca778b-35f9-41e3-bdf1-f51b3d29f220/volumes" Jun 15 09:26:02.463866 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:02.463806 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:02.564009 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:02.563973 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:02.666806 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:02.666770 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:04.912583 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:04.912547 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:05.012811 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:05.012775 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:07.611570 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.611531 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx"] Jun 15 09:26:07.612022 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.611961 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89ca778b-35f9-41e3-bdf1-f51b3d29f220" containerName="manager" Jun 15 09:26:07.612022 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.611976 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="89ca778b-35f9-41e3-bdf1-f51b3d29f220" containerName="manager" Jun 15 09:26:07.612132 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.612077 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="89ca778b-35f9-41e3-bdf1-f51b3d29f220" containerName="manager" Jun 15 09:26:07.630273 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.630237 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:07.664242 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.664209 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx"] Jun 15 09:26:07.664461 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.664315 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.667197 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.666830 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-sb8fb\"" Jun 15 09:26:07.667197 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.666935 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 15 09:26:07.797648 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.797607 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30d60e63-17db-47e9-800c-2c5bd368678e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.797648 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.797651 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30d60e63-17db-47e9-800c-2c5bd368678e-workload-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.797872 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.797753 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30d60e63-17db-47e9-800c-2c5bd368678e-istio-data\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.797872 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.797796 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30d60e63-17db-47e9-800c-2c5bd368678e-istio-token\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.797872 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.797821 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldtg9\" (UniqueName: \"kubernetes.io/projected/30d60e63-17db-47e9-800c-2c5bd368678e-kube-api-access-ldtg9\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.797872 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.797861 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30d60e63-17db-47e9-800c-2c5bd368678e-workload-certs\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.798004 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.797877 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30d60e63-17db-47e9-800c-2c5bd368678e-istio-envoy\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.798004 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.797899 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30d60e63-17db-47e9-800c-2c5bd368678e-credential-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.798004 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.797921 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30d60e63-17db-47e9-800c-2c5bd368678e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.899142 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.899058 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30d60e63-17db-47e9-800c-2c5bd368678e-istio-data\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.899142 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.899109 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30d60e63-17db-47e9-800c-2c5bd368678e-istio-token\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.899142 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.899138 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldtg9\" (UniqueName: \"kubernetes.io/projected/30d60e63-17db-47e9-800c-2c5bd368678e-kube-api-access-ldtg9\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.899418 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.899169 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30d60e63-17db-47e9-800c-2c5bd368678e-workload-certs\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.899418 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.899195 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30d60e63-17db-47e9-800c-2c5bd368678e-istio-envoy\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.899418 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.899226 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30d60e63-17db-47e9-800c-2c5bd368678e-credential-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.899418 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.899260 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30d60e63-17db-47e9-800c-2c5bd368678e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.899418 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.899322 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30d60e63-17db-47e9-800c-2c5bd368678e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.899418 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.899349 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30d60e63-17db-47e9-800c-2c5bd368678e-workload-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.899798 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.899774 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30d60e63-17db-47e9-800c-2c5bd368678e-workload-certs\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.899959 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.899934 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30d60e63-17db-47e9-800c-2c5bd368678e-workload-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.899959 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.899828 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30d60e63-17db-47e9-800c-2c5bd368678e-credential-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.900159 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.900141 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30d60e63-17db-47e9-800c-2c5bd368678e-istio-data\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.900218 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.900191 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30d60e63-17db-47e9-800c-2c5bd368678e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.901861 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.901818 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30d60e63-17db-47e9-800c-2c5bd368678e-istio-envoy\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.901991 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.901971 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30d60e63-17db-47e9-800c-2c5bd368678e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.906787 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.906767 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30d60e63-17db-47e9-800c-2c5bd368678e-istio-token\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.907111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.907090 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldtg9\" (UniqueName: \"kubernetes.io/projected/30d60e63-17db-47e9-800c-2c5bd368678e-kube-api-access-ldtg9\") pod \"maas-default-gateway-openshift-default-8559cd5744-qxvtx\" (UID: \"30d60e63-17db-47e9-800c-2c5bd368678e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:07.975571 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:07.975525 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:08.131692 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:08.131668 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx"] Jun 15 09:26:08.134620 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:26:08.134587 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30d60e63_17db_47e9_800c_2c5bd368678e.slice/crio-497d3858969b4b8015ba0c6352b3b3a54f48a1caddabd7914ae80f96f954be06 WatchSource:0}: Error finding container 497d3858969b4b8015ba0c6352b3b3a54f48a1caddabd7914ae80f96f954be06: Status 404 returned error can't find the container with id 497d3858969b4b8015ba0c6352b3b3a54f48a1caddabd7914ae80f96f954be06 Jun 15 09:26:08.528252 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:08.528213 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:08.687978 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:08.687944 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" event={"ID":"30d60e63-17db-47e9-800c-2c5bd368678e","Type":"ContainerStarted","Data":"497d3858969b4b8015ba0c6352b3b3a54f48a1caddabd7914ae80f96f954be06"} Jun 15 09:26:09.664648 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:09.664603 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:09.762318 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:09.762280 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:10.745258 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:10.745214 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 15 09:26:10.745365 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:10.745291 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 15 09:26:10.745365 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:10.745326 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 15 09:26:11.704290 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:11.704257 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" event={"ID":"30d60e63-17db-47e9-800c-2c5bd368678e","Type":"ContainerStarted","Data":"6cbe5ca5371e6a7184546156adc6ff05affa6d7d0e5c7d4bf3ad6a948c852d0b"} Jun 15 09:26:11.724909 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:11.724864 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" podStartSLOduration=2.117091826 podStartE2EDuration="4.724848205s" totalImagesPullingTime="2.607756379s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:26:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:26:11.72273122 +0000 UTC m=+456.286153526" watchObservedRunningTime="2026-06-15 09:26:11.724848205 +0000 UTC m=+456.288270501" Jun 15 09:26:11.976036 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:11.975944 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:11.980696 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:11.980672 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:12.708903 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:12.708870 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:12.709987 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:12.709965 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:26:12.851414 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:12.851386 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-psfc5"] Jun 15 09:26:12.854077 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:12.854051 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:12.854425 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:12.854410 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-psfc5" Jun 15 09:26:12.856873 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:12.856852 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 15 09:26:12.868609 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:12.868586 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-psfc5"] Jun 15 09:26:12.943989 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:12.943960 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vlrt\" (UniqueName: \"kubernetes.io/projected/37cf95d1-efa8-4cd4-90fa-f3cdd5074764-kube-api-access-2vlrt\") pod \"limitador-limitador-64c94fd64f-psfc5\" (UID: \"37cf95d1-efa8-4cd4-90fa-f3cdd5074764\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-psfc5" Jun 15 09:26:12.944143 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:12.943998 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/37cf95d1-efa8-4cd4-90fa-f3cdd5074764-config-file\") pod \"limitador-limitador-64c94fd64f-psfc5\" (UID: \"37cf95d1-efa8-4cd4-90fa-f3cdd5074764\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-psfc5" Jun 15 09:26:12.952744 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:12.952706 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-psfc5"] Jun 15 09:26:12.953596 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:12.953565 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:13.021485 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:13.021397 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:13.045520 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:13.045489 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vlrt\" (UniqueName: \"kubernetes.io/projected/37cf95d1-efa8-4cd4-90fa-f3cdd5074764-kube-api-access-2vlrt\") pod \"limitador-limitador-64c94fd64f-psfc5\" (UID: \"37cf95d1-efa8-4cd4-90fa-f3cdd5074764\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-psfc5" Jun 15 09:26:13.045609 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:13.045528 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/37cf95d1-efa8-4cd4-90fa-f3cdd5074764-config-file\") pod \"limitador-limitador-64c94fd64f-psfc5\" (UID: \"37cf95d1-efa8-4cd4-90fa-f3cdd5074764\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-psfc5" Jun 15 09:26:13.046237 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:13.046217 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/37cf95d1-efa8-4cd4-90fa-f3cdd5074764-config-file\") pod \"limitador-limitador-64c94fd64f-psfc5\" (UID: \"37cf95d1-efa8-4cd4-90fa-f3cdd5074764\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-psfc5" Jun 15 09:26:13.053522 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:13.053501 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vlrt\" (UniqueName: \"kubernetes.io/projected/37cf95d1-efa8-4cd4-90fa-f3cdd5074764-kube-api-access-2vlrt\") pod \"limitador-limitador-64c94fd64f-psfc5\" (UID: \"37cf95d1-efa8-4cd4-90fa-f3cdd5074764\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-psfc5" Jun 15 09:26:13.163670 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:13.163625 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-psfc5" Jun 15 09:26:13.284482 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:13.284454 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-psfc5"] Jun 15 09:26:13.286965 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:26:13.286937 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37cf95d1_efa8_4cd4_90fa_f3cdd5074764.slice/crio-70c5a670ff7546bde44e8ea79f7b271d6b7015376a8c6bbddcc36fae4ce9170a WatchSource:0}: Error finding container 70c5a670ff7546bde44e8ea79f7b271d6b7015376a8c6bbddcc36fae4ce9170a: Status 404 returned error can't find the container with id 70c5a670ff7546bde44e8ea79f7b271d6b7015376a8c6bbddcc36fae4ce9170a Jun 15 09:26:13.666038 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:13.665997 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:13.713688 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:13.713652 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-psfc5" event={"ID":"37cf95d1-efa8-4cd4-90fa-f3cdd5074764","Type":"ContainerStarted","Data":"70c5a670ff7546bde44e8ea79f7b271d6b7015376a8c6bbddcc36fae4ce9170a"} Jun 15 09:26:13.725296 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:13.725251 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:13.891618 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:13.891533 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:13.993196 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:13.993040 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:16.729699 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:16.729663 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-psfc5" event={"ID":"37cf95d1-efa8-4cd4-90fa-f3cdd5074764","Type":"ContainerStarted","Data":"7de4ced3c0569c0f5754763c32bf36607960dd1acf7a6aaca50a53abe2a0dc5b"} Jun 15 09:26:16.730129 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:16.729805 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-psfc5" Jun 15 09:26:16.746299 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:16.746255 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-psfc5" podStartSLOduration=2.078831419 podStartE2EDuration="4.746239934s" totalImagesPullingTime="2.667408515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:26:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:26:16.744684013 +0000 UTC m=+461.308106317" watchObservedRunningTime="2026-06-15 09:26:16.746239934 +0000 UTC m=+461.309662282" Jun 15 09:26:22.686008 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:22.685967 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:22.780525 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:22.780492 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:25.139007 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:25.138970 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:25.237170 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:25.237139 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:27.734542 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:27.734510 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-psfc5" Jun 15 09:26:27.801889 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:27.801859 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-psfc5"] Jun 15 09:26:27.802037 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:27.802008 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-psfc5" podUID="37cf95d1-efa8-4cd4-90fa-f3cdd5074764" containerName="limitador" containerID="cri-o://7de4ced3c0569c0f5754763c32bf36607960dd1acf7a6aaca50a53abe2a0dc5b" gracePeriod=30 Jun 15 09:26:28.337783 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:28.337754 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-psfc5" Jun 15 09:26:28.483884 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:28.483790 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/37cf95d1-efa8-4cd4-90fa-f3cdd5074764-config-file\") pod \"37cf95d1-efa8-4cd4-90fa-f3cdd5074764\" (UID: \"37cf95d1-efa8-4cd4-90fa-f3cdd5074764\") " Jun 15 09:26:28.484026 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:28.483886 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vlrt\" (UniqueName: \"kubernetes.io/projected/37cf95d1-efa8-4cd4-90fa-f3cdd5074764-kube-api-access-2vlrt\") pod \"37cf95d1-efa8-4cd4-90fa-f3cdd5074764\" (UID: \"37cf95d1-efa8-4cd4-90fa-f3cdd5074764\") " Jun 15 09:26:28.484242 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:28.484212 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37cf95d1-efa8-4cd4-90fa-f3cdd5074764-config-file" (OuterVolumeSpecName: "config-file") pod "37cf95d1-efa8-4cd4-90fa-f3cdd5074764" (UID: "37cf95d1-efa8-4cd4-90fa-f3cdd5074764"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 09:26:28.485923 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:28.485906 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37cf95d1-efa8-4cd4-90fa-f3cdd5074764-kube-api-access-2vlrt" (OuterVolumeSpecName: "kube-api-access-2vlrt") pod "37cf95d1-efa8-4cd4-90fa-f3cdd5074764" (UID: "37cf95d1-efa8-4cd4-90fa-f3cdd5074764"). InnerVolumeSpecName "kube-api-access-2vlrt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 09:26:28.585155 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:28.585123 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vlrt\" (UniqueName: \"kubernetes.io/projected/37cf95d1-efa8-4cd4-90fa-f3cdd5074764-kube-api-access-2vlrt\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:26:28.585155 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:28.585148 2562 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/37cf95d1-efa8-4cd4-90fa-f3cdd5074764-config-file\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:26:28.778886 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:28.778789 2562 generic.go:356] "Generic (PLEG): container finished" podID="37cf95d1-efa8-4cd4-90fa-f3cdd5074764" containerID="7de4ced3c0569c0f5754763c32bf36607960dd1acf7a6aaca50a53abe2a0dc5b" exitCode=0 Jun 15 09:26:28.778886 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:28.778848 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-psfc5" Jun 15 09:26:28.778886 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:28.778870 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-psfc5" event={"ID":"37cf95d1-efa8-4cd4-90fa-f3cdd5074764","Type":"ContainerDied","Data":"7de4ced3c0569c0f5754763c32bf36607960dd1acf7a6aaca50a53abe2a0dc5b"} Jun 15 09:26:28.779343 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:28.778917 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-psfc5" event={"ID":"37cf95d1-efa8-4cd4-90fa-f3cdd5074764","Type":"ContainerDied","Data":"70c5a670ff7546bde44e8ea79f7b271d6b7015376a8c6bbddcc36fae4ce9170a"} Jun 15 09:26:28.779343 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:28.778939 2562 scope.go:117] "RemoveContainer" containerID="7de4ced3c0569c0f5754763c32bf36607960dd1acf7a6aaca50a53abe2a0dc5b" Jun 15 09:26:28.787646 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:28.787632 2562 scope.go:117] "RemoveContainer" containerID="7de4ced3c0569c0f5754763c32bf36607960dd1acf7a6aaca50a53abe2a0dc5b" Jun 15 09:26:28.787899 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:26:28.787883 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7de4ced3c0569c0f5754763c32bf36607960dd1acf7a6aaca50a53abe2a0dc5b\": container with ID starting with 7de4ced3c0569c0f5754763c32bf36607960dd1acf7a6aaca50a53abe2a0dc5b not found: ID does not exist" containerID="7de4ced3c0569c0f5754763c32bf36607960dd1acf7a6aaca50a53abe2a0dc5b" Jun 15 09:26:28.787968 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:28.787905 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7de4ced3c0569c0f5754763c32bf36607960dd1acf7a6aaca50a53abe2a0dc5b"} err="failed to get container status \"7de4ced3c0569c0f5754763c32bf36607960dd1acf7a6aaca50a53abe2a0dc5b\": rpc error: code = NotFound desc = could not find container \"7de4ced3c0569c0f5754763c32bf36607960dd1acf7a6aaca50a53abe2a0dc5b\": container with ID starting with 7de4ced3c0569c0f5754763c32bf36607960dd1acf7a6aaca50a53abe2a0dc5b not found: ID does not exist" Jun 15 09:26:28.798875 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:28.798851 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-psfc5"] Jun 15 09:26:28.802325 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:28.802304 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-psfc5"] Jun 15 09:26:29.436252 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:29.436215 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-6p2lm"] Jun 15 09:26:29.436936 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:29.436911 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37cf95d1-efa8-4cd4-90fa-f3cdd5074764" containerName="limitador" Jun 15 09:26:29.436936 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:29.436936 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="37cf95d1-efa8-4cd4-90fa-f3cdd5074764" containerName="limitador" Jun 15 09:26:29.437093 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:29.437076 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="37cf95d1-efa8-4cd4-90fa-f3cdd5074764" containerName="limitador" Jun 15 09:26:29.442429 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:29.442406 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-6p2lm" Jun 15 09:26:29.444614 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:29.444589 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-6p2lm"] Jun 15 09:26:29.445380 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:29.445353 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jun 15 09:26:29.445508 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:29.445357 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-hvbwf\"" Jun 15 09:26:29.465880 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:29.465856 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:29.592818 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:29.592788 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6f42c62e-7931-4fb9-857f-b334967d9e9a-data\") pod \"postgres-69fb4594cf-6p2lm\" (UID: \"6f42c62e-7931-4fb9-857f-b334967d9e9a\") " pod="opendatahub/postgres-69fb4594cf-6p2lm" Jun 15 09:26:29.593031 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:29.592823 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgt2q\" (UniqueName: \"kubernetes.io/projected/6f42c62e-7931-4fb9-857f-b334967d9e9a-kube-api-access-cgt2q\") pod \"postgres-69fb4594cf-6p2lm\" (UID: \"6f42c62e-7931-4fb9-857f-b334967d9e9a\") " pod="opendatahub/postgres-69fb4594cf-6p2lm" Jun 15 09:26:29.693739 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:29.693663 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6f42c62e-7931-4fb9-857f-b334967d9e9a-data\") pod \"postgres-69fb4594cf-6p2lm\" (UID: \"6f42c62e-7931-4fb9-857f-b334967d9e9a\") " pod="opendatahub/postgres-69fb4594cf-6p2lm" Jun 15 09:26:29.693739 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:29.693705 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgt2q\" (UniqueName: \"kubernetes.io/projected/6f42c62e-7931-4fb9-857f-b334967d9e9a-kube-api-access-cgt2q\") pod \"postgres-69fb4594cf-6p2lm\" (UID: \"6f42c62e-7931-4fb9-857f-b334967d9e9a\") " pod="opendatahub/postgres-69fb4594cf-6p2lm" Jun 15 09:26:29.694185 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:29.694168 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6f42c62e-7931-4fb9-857f-b334967d9e9a-data\") pod \"postgres-69fb4594cf-6p2lm\" (UID: \"6f42c62e-7931-4fb9-857f-b334967d9e9a\") " pod="opendatahub/postgres-69fb4594cf-6p2lm" Jun 15 09:26:29.704273 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:29.704244 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgt2q\" (UniqueName: \"kubernetes.io/projected/6f42c62e-7931-4fb9-857f-b334967d9e9a-kube-api-access-cgt2q\") pod \"postgres-69fb4594cf-6p2lm\" (UID: \"6f42c62e-7931-4fb9-857f-b334967d9e9a\") " pod="opendatahub/postgres-69fb4594cf-6p2lm" Jun 15 09:26:29.754670 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:29.754630 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-6p2lm" Jun 15 09:26:29.880628 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:29.880585 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-6p2lm"] Jun 15 09:26:29.881241 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:29.881160 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:29.883580 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:26:29.883557 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f42c62e_7931_4fb9_857f_b334967d9e9a.slice/crio-6a1156ebfcc733ba424e45be3e9db92cf72390c2f5e55e5055d9fb991974130f WatchSource:0}: Error finding container 6a1156ebfcc733ba424e45be3e9db92cf72390c2f5e55e5055d9fb991974130f: Status 404 returned error can't find the container with id 6a1156ebfcc733ba424e45be3e9db92cf72390c2f5e55e5055d9fb991974130f Jun 15 09:26:29.935261 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:29.935237 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37cf95d1-efa8-4cd4-90fa-f3cdd5074764" path="/var/lib/kubelet/pods/37cf95d1-efa8-4cd4-90fa-f3cdd5074764/volumes" Jun 15 09:26:29.982073 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:29.982014 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:30.788917 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:30.788885 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-6p2lm" event={"ID":"6f42c62e-7931-4fb9-857f-b334967d9e9a","Type":"ContainerStarted","Data":"6a1156ebfcc733ba424e45be3e9db92cf72390c2f5e55e5055d9fb991974130f"} Jun 15 09:26:32.800397 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:32.800358 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-6p2lm" event={"ID":"6f42c62e-7931-4fb9-857f-b334967d9e9a","Type":"ContainerStarted","Data":"5960aa38f2603b79947c2cf35d899c93a4c04fae048a7a4c038eb854e5674123"} Jun 15 09:26:32.800756 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:32.800468 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-6p2lm" Jun 15 09:26:32.814883 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:32.814817 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-6p2lm" podStartSLOduration=1.086245463 podStartE2EDuration="3.814803883s" totalImagesPullingTime="2.72855842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:26:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:26:32.8137966 +0000 UTC m=+477.377218924" watchObservedRunningTime="2026-06-15 09:26:32.814803883 +0000 UTC m=+477.378226188" Jun 15 09:26:38.836738 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:38.836704 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-6p2lm" Jun 15 09:26:39.158760 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:39.158679 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:39.261676 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:39.261640 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:26:39.396740 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:26:39.396706 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:27:01.593633 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:01.593601 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:27:01.695072 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:01.695039 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:27:02.002493 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:02.002417 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 15 09:27:02.005788 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:02.005755 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 15 09:27:02.008465 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:02.008443 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 15 09:27:02.008588 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:02.008484 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 15 09:27:02.008588 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:02.008549 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 15 09:27:02.009396 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:02.009379 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-mmbq9\"" Jun 15 09:27:02.017684 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:02.017665 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 15 09:27:02.050600 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:02.050573 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t94nw\" (UniqueName: \"kubernetes.io/projected/eec632c6-82b7-44c9-b493-c1b36fb7a53f-kube-api-access-t94nw\") pod \"maas-keycloak-0\" (UID: \"eec632c6-82b7-44c9-b493-c1b36fb7a53f\") " pod="keycloak-system/maas-keycloak-0" Jun 15 09:27:02.151568 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:02.151540 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t94nw\" (UniqueName: \"kubernetes.io/projected/eec632c6-82b7-44c9-b493-c1b36fb7a53f-kube-api-access-t94nw\") pod \"maas-keycloak-0\" (UID: \"eec632c6-82b7-44c9-b493-c1b36fb7a53f\") " pod="keycloak-system/maas-keycloak-0" Jun 15 09:27:02.159950 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:02.159929 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t94nw\" (UniqueName: \"kubernetes.io/projected/eec632c6-82b7-44c9-b493-c1b36fb7a53f-kube-api-access-t94nw\") pod \"maas-keycloak-0\" (UID: \"eec632c6-82b7-44c9-b493-c1b36fb7a53f\") " pod="keycloak-system/maas-keycloak-0" Jun 15 09:27:02.315204 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:02.315133 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 15 09:27:02.439187 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:02.439154 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 15 09:27:02.443261 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:27:02.443229 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeec632c6_82b7_44c9_b493_c1b36fb7a53f.slice/crio-4dbaaa8d03b7f7a127a25f5da2c91d849607839870d7d13c6194ee57c7c7cb57 WatchSource:0}: Error finding container 4dbaaa8d03b7f7a127a25f5da2c91d849607839870d7d13c6194ee57c7c7cb57: Status 404 returned error can't find the container with id 4dbaaa8d03b7f7a127a25f5da2c91d849607839870d7d13c6194ee57c7c7cb57 Jun 15 09:27:02.920786 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:02.920689 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"eec632c6-82b7-44c9-b493-c1b36fb7a53f","Type":"ContainerStarted","Data":"4dbaaa8d03b7f7a127a25f5da2c91d849607839870d7d13c6194ee57c7c7cb57"} Jun 15 09:27:06.086096 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:06.086058 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:27:06.135894 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:06.135829 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:27:07.947211 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:07.947168 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"eec632c6-82b7-44c9-b493-c1b36fb7a53f","Type":"ContainerStarted","Data":"8cd39a7c15b1c114bca29cf82b30e617982120842e3b82a1dc63190b00d35674"} Jun 15 09:27:07.966785 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:07.966731 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=2.366074941 podStartE2EDuration="6.966712581s" totalImagesPullingTime="4.60063764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:27:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:27:07.965368673 +0000 UTC m=+512.528790979" watchObservedRunningTime="2026-06-15 09:27:07.966712581 +0000 UTC m=+512.530134886" Jun 15 09:27:08.315905 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:08.315787 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 15 09:27:08.317804 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:08.317751 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eec632c6-82b7-44c9-b493-c1b36fb7a53f" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.26:9000/health/started\": dial tcp 10.133.0.26:9000: connect: connection refused" Jun 15 09:27:09.315988 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:09.315942 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eec632c6-82b7-44c9-b493-c1b36fb7a53f" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.26:9000/health/started\": dial tcp 10.133.0.26:9000: connect: connection refused" Jun 15 09:27:10.316156 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:10.316107 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eec632c6-82b7-44c9-b493-c1b36fb7a53f" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.26:9000/health/started\": dial tcp 10.133.0.26:9000: connect: connection refused" Jun 15 09:27:11.316124 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:11.316073 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eec632c6-82b7-44c9-b493-c1b36fb7a53f" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.26:9000/health/started\": dial tcp 10.133.0.26:9000: connect: connection refused" Jun 15 09:27:12.315937 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:12.315263 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 15 09:27:12.315937 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:12.315708 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eec632c6-82b7-44c9-b493-c1b36fb7a53f" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.26:9000/health/started\": dial tcp 10.133.0.26:9000: connect: connection refused" Jun 15 09:27:13.316699 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:13.316645 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eec632c6-82b7-44c9-b493-c1b36fb7a53f" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.26:9000/health/started\": dial tcp 10.133.0.26:9000: connect: connection refused" Jun 15 09:27:14.316353 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:14.315943 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eec632c6-82b7-44c9-b493-c1b36fb7a53f" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.26:9000/health/started\": dial tcp 10.133.0.26:9000: connect: connection refused" Jun 15 09:27:15.316568 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:15.316515 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eec632c6-82b7-44c9-b493-c1b36fb7a53f" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.26:9000/health/started\": dial tcp 10.133.0.26:9000: connect: connection refused" Jun 15 09:27:16.316180 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:16.316133 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eec632c6-82b7-44c9-b493-c1b36fb7a53f" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.26:9000/health/started\": dial tcp 10.133.0.26:9000: connect: connection refused" Jun 15 09:27:17.315798 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:17.315751 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eec632c6-82b7-44c9-b493-c1b36fb7a53f" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.26:9000/health/started\": dial tcp 10.133.0.26:9000: connect: connection refused" Jun 15 09:27:18.316196 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:18.316141 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eec632c6-82b7-44c9-b493-c1b36fb7a53f" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.26:9000/health/started\": dial tcp 10.133.0.26:9000: connect: connection refused" Jun 15 09:27:19.316190 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:19.316143 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eec632c6-82b7-44c9-b493-c1b36fb7a53f" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.26:9000/health/started\": dial tcp 10.133.0.26:9000: connect: connection refused" Jun 15 09:27:20.404692 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:20.404639 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 15 09:27:20.423125 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:20.423083 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="eec632c6-82b7-44c9-b493-c1b36fb7a53f" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 15 09:27:30.411006 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:30.410911 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 15 09:27:31.036746 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:31.036699 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:27:31.378281 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:31.378243 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:27:36.185713 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.185677 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:27:36.198345 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.198319 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-776b74d8c5-v9fnj"] Jun 15 09:27:36.203753 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.203736 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-776b74d8c5-v9fnj" Jun 15 09:27:36.206721 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.206671 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 15 09:27:36.206876 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.206717 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 15 09:27:36.206876 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.206737 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5njw7\"" Jun 15 09:27:36.213397 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.213373 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-776b74d8c5-v9fnj"] Jun 15 09:27:36.274430 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.274402 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdz4t\" (UniqueName: \"kubernetes.io/projected/68493623-e470-46ff-ad9d-e5e51fde8162-kube-api-access-fdz4t\") pod \"maas-controller-776b74d8c5-v9fnj\" (UID: \"68493623-e470-46ff-ad9d-e5e51fde8162\") " pod="opendatahub/maas-controller-776b74d8c5-v9fnj" Jun 15 09:27:36.274598 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.274439 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68493623-e470-46ff-ad9d-e5e51fde8162-webhook-cert\") pod \"maas-controller-776b74d8c5-v9fnj\" (UID: \"68493623-e470-46ff-ad9d-e5e51fde8162\") " pod="opendatahub/maas-controller-776b74d8c5-v9fnj" Jun 15 09:27:36.286963 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.286932 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:27:36.361713 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.361680 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:27:36.366344 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.366311 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-767f9856ff-sxbgt"] Jun 15 09:27:36.370226 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.370205 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-767f9856ff-sxbgt" Jun 15 09:27:36.376185 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.376155 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdz4t\" (UniqueName: \"kubernetes.io/projected/68493623-e470-46ff-ad9d-e5e51fde8162-kube-api-access-fdz4t\") pod \"maas-controller-776b74d8c5-v9fnj\" (UID: \"68493623-e470-46ff-ad9d-e5e51fde8162\") " pod="opendatahub/maas-controller-776b74d8c5-v9fnj" Jun 15 09:27:36.376337 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.376201 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68493623-e470-46ff-ad9d-e5e51fde8162-webhook-cert\") pod \"maas-controller-776b74d8c5-v9fnj\" (UID: \"68493623-e470-46ff-ad9d-e5e51fde8162\") " pod="opendatahub/maas-controller-776b74d8c5-v9fnj" Jun 15 09:27:36.376397 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:27:36.376350 2562 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jun 15 09:27:36.376447 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:27:36.376398 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68493623-e470-46ff-ad9d-e5e51fde8162-webhook-cert podName:68493623-e470-46ff-ad9d-e5e51fde8162 nodeName:}" failed. No retries permitted until 2026-06-15 09:27:36.876381785 +0000 UTC m=+541.439804069 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/68493623-e470-46ff-ad9d-e5e51fde8162-webhook-cert") pod "maas-controller-776b74d8c5-v9fnj" (UID: "68493623-e470-46ff-ad9d-e5e51fde8162") : secret "maas-controller-webhook-cert" not found Jun 15 09:27:36.378675 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.378649 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-767f9856ff-sxbgt"] Jun 15 09:27:36.386983 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.386955 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdz4t\" (UniqueName: \"kubernetes.io/projected/68493623-e470-46ff-ad9d-e5e51fde8162-kube-api-access-fdz4t\") pod \"maas-controller-776b74d8c5-v9fnj\" (UID: \"68493623-e470-46ff-ad9d-e5e51fde8162\") " pod="opendatahub/maas-controller-776b74d8c5-v9fnj" Jun 15 09:27:36.477220 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.477127 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/85021cfa-1559-4ece-81f3-556fe5004e5a-webhook-cert\") pod \"maas-controller-767f9856ff-sxbgt\" (UID: \"85021cfa-1559-4ece-81f3-556fe5004e5a\") " pod="opendatahub/maas-controller-767f9856ff-sxbgt" Jun 15 09:27:36.477220 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.477164 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddw8g\" (UniqueName: \"kubernetes.io/projected/85021cfa-1559-4ece-81f3-556fe5004e5a-kube-api-access-ddw8g\") pod \"maas-controller-767f9856ff-sxbgt\" (UID: \"85021cfa-1559-4ece-81f3-556fe5004e5a\") " pod="opendatahub/maas-controller-767f9856ff-sxbgt" Jun 15 09:27:36.578210 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.578176 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/85021cfa-1559-4ece-81f3-556fe5004e5a-webhook-cert\") pod \"maas-controller-767f9856ff-sxbgt\" (UID: \"85021cfa-1559-4ece-81f3-556fe5004e5a\") " pod="opendatahub/maas-controller-767f9856ff-sxbgt" Jun 15 09:27:36.578210 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.578211 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddw8g\" (UniqueName: \"kubernetes.io/projected/85021cfa-1559-4ece-81f3-556fe5004e5a-kube-api-access-ddw8g\") pod \"maas-controller-767f9856ff-sxbgt\" (UID: \"85021cfa-1559-4ece-81f3-556fe5004e5a\") " pod="opendatahub/maas-controller-767f9856ff-sxbgt" Jun 15 09:27:36.580509 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.580487 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/85021cfa-1559-4ece-81f3-556fe5004e5a-webhook-cert\") pod \"maas-controller-767f9856ff-sxbgt\" (UID: \"85021cfa-1559-4ece-81f3-556fe5004e5a\") " pod="opendatahub/maas-controller-767f9856ff-sxbgt" Jun 15 09:27:36.585730 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.585706 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddw8g\" (UniqueName: \"kubernetes.io/projected/85021cfa-1559-4ece-81f3-556fe5004e5a-kube-api-access-ddw8g\") pod \"maas-controller-767f9856ff-sxbgt\" (UID: \"85021cfa-1559-4ece-81f3-556fe5004e5a\") " pod="opendatahub/maas-controller-767f9856ff-sxbgt" Jun 15 09:27:36.589448 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.589426 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:27:36.681294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.681268 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-767f9856ff-sxbgt" Jun 15 09:27:36.812213 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.812188 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-767f9856ff-sxbgt"] Jun 15 09:27:36.814375 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:27:36.814349 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85021cfa_1559_4ece_81f3_556fe5004e5a.slice/crio-6e544544ee29158c3893e80a2db535449c47cca420cfcbc2a406d7b339f72694 WatchSource:0}: Error finding container 6e544544ee29158c3893e80a2db535449c47cca420cfcbc2a406d7b339f72694: Status 404 returned error can't find the container with id 6e544544ee29158c3893e80a2db535449c47cca420cfcbc2a406d7b339f72694 Jun 15 09:27:36.880990 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.880962 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68493623-e470-46ff-ad9d-e5e51fde8162-webhook-cert\") pod \"maas-controller-776b74d8c5-v9fnj\" (UID: \"68493623-e470-46ff-ad9d-e5e51fde8162\") " pod="opendatahub/maas-controller-776b74d8c5-v9fnj" Jun 15 09:27:36.883237 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:36.883217 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68493623-e470-46ff-ad9d-e5e51fde8162-webhook-cert\") pod \"maas-controller-776b74d8c5-v9fnj\" (UID: \"68493623-e470-46ff-ad9d-e5e51fde8162\") " pod="opendatahub/maas-controller-776b74d8c5-v9fnj" Jun 15 09:27:37.089610 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:37.089576 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-767f9856ff-sxbgt" event={"ID":"85021cfa-1559-4ece-81f3-556fe5004e5a","Type":"ContainerStarted","Data":"6e544544ee29158c3893e80a2db535449c47cca420cfcbc2a406d7b339f72694"} Jun 15 09:27:37.114767 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:37.114737 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-776b74d8c5-v9fnj" Jun 15 09:27:37.241784 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:37.241749 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-776b74d8c5-v9fnj"] Jun 15 09:27:37.244122 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:27:37.244095 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68493623_e470_46ff_ad9d_e5e51fde8162.slice/crio-6cadf62e759fed8a6505302a74531b217cae2abe786e3026f4967e6b374affdc WatchSource:0}: Error finding container 6cadf62e759fed8a6505302a74531b217cae2abe786e3026f4967e6b374affdc: Status 404 returned error can't find the container with id 6cadf62e759fed8a6505302a74531b217cae2abe786e3026f4967e6b374affdc Jun 15 09:27:38.096311 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:38.096265 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-776b74d8c5-v9fnj" event={"ID":"68493623-e470-46ff-ad9d-e5e51fde8162","Type":"ContainerStarted","Data":"6cadf62e759fed8a6505302a74531b217cae2abe786e3026f4967e6b374affdc"} Jun 15 09:27:38.853661 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:38.853625 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:27:38.987569 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:38.987471 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:27:39.101589 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:39.101548 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-767f9856ff-sxbgt" event={"ID":"85021cfa-1559-4ece-81f3-556fe5004e5a","Type":"ContainerStarted","Data":"a9f750d6fd7317bbec2246cc6b082fd4984536d45db3cd64ecdd12b4359ec29a"} Jun 15 09:27:39.101792 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:39.101615 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-767f9856ff-sxbgt" Jun 15 09:27:39.103005 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:39.102982 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-776b74d8c5-v9fnj" event={"ID":"68493623-e470-46ff-ad9d-e5e51fde8162","Type":"ContainerStarted","Data":"58ccb9eae766ed1e79267218970231f43637920910687c22dccdd1ce3d789933"} Jun 15 09:27:39.103134 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:39.103117 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-776b74d8c5-v9fnj" Jun 15 09:27:39.123089 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:39.123039 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-767f9856ff-sxbgt" podStartSLOduration=1.267905824 podStartE2EDuration="3.123023022s" totalImagesPullingTime="1.855117198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:27:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:27:39.122906918 +0000 UTC m=+543.686329224" watchObservedRunningTime="2026-06-15 09:27:39.123023022 +0000 UTC m=+543.686445328" Jun 15 09:27:39.144887 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:39.144815 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-776b74d8c5-v9fnj" podStartSLOduration=1.715522443 podStartE2EDuration="3.144800561s" totalImagesPullingTime="1.429278118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:27:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:27:39.143864917 +0000 UTC m=+543.707287221" watchObservedRunningTime="2026-06-15 09:27:39.144800561 +0000 UTC m=+543.708222866" Jun 15 09:27:50.112208 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:50.112178 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-767f9856ff-sxbgt" Jun 15 09:27:50.112694 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:50.112329 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-776b74d8c5-v9fnj" Jun 15 09:27:50.174241 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:50.174206 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-776b74d8c5-v9fnj"] Jun 15 09:27:50.174460 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:50.174436 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-776b74d8c5-v9fnj" podUID="68493623-e470-46ff-ad9d-e5e51fde8162" containerName="manager" containerID="cri-o://58ccb9eae766ed1e79267218970231f43637920910687c22dccdd1ce3d789933" gracePeriod=30 Jun 15 09:27:50.419865 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:50.419820 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-776b74d8c5-v9fnj" Jun 15 09:27:50.495606 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:50.495582 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdz4t\" (UniqueName: \"kubernetes.io/projected/68493623-e470-46ff-ad9d-e5e51fde8162-kube-api-access-fdz4t\") pod \"68493623-e470-46ff-ad9d-e5e51fde8162\" (UID: \"68493623-e470-46ff-ad9d-e5e51fde8162\") " Jun 15 09:27:50.495766 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:50.495685 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68493623-e470-46ff-ad9d-e5e51fde8162-webhook-cert\") pod \"68493623-e470-46ff-ad9d-e5e51fde8162\" (UID: \"68493623-e470-46ff-ad9d-e5e51fde8162\") " Jun 15 09:27:50.497735 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:50.497704 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68493623-e470-46ff-ad9d-e5e51fde8162-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "68493623-e470-46ff-ad9d-e5e51fde8162" (UID: "68493623-e470-46ff-ad9d-e5e51fde8162"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 09:27:50.497889 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:50.497789 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68493623-e470-46ff-ad9d-e5e51fde8162-kube-api-access-fdz4t" (OuterVolumeSpecName: "kube-api-access-fdz4t") pod "68493623-e470-46ff-ad9d-e5e51fde8162" (UID: "68493623-e470-46ff-ad9d-e5e51fde8162"). InnerVolumeSpecName "kube-api-access-fdz4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 09:27:50.596284 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:50.596251 2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68493623-e470-46ff-ad9d-e5e51fde8162-webhook-cert\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:27:50.596284 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:50.596280 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdz4t\" (UniqueName: \"kubernetes.io/projected/68493623-e470-46ff-ad9d-e5e51fde8162-kube-api-access-fdz4t\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:27:51.158292 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:51.158256 2562 generic.go:356] "Generic (PLEG): container finished" podID="68493623-e470-46ff-ad9d-e5e51fde8162" containerID="58ccb9eae766ed1e79267218970231f43637920910687c22dccdd1ce3d789933" exitCode=0 Jun 15 09:27:51.158737 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:51.158316 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-776b74d8c5-v9fnj" Jun 15 09:27:51.158737 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:51.158325 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-776b74d8c5-v9fnj" event={"ID":"68493623-e470-46ff-ad9d-e5e51fde8162","Type":"ContainerDied","Data":"58ccb9eae766ed1e79267218970231f43637920910687c22dccdd1ce3d789933"} Jun 15 09:27:51.158737 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:51.158412 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-776b74d8c5-v9fnj" event={"ID":"68493623-e470-46ff-ad9d-e5e51fde8162","Type":"ContainerDied","Data":"6cadf62e759fed8a6505302a74531b217cae2abe786e3026f4967e6b374affdc"} Jun 15 09:27:51.158737 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:51.158446 2562 scope.go:117] "RemoveContainer" containerID="58ccb9eae766ed1e79267218970231f43637920910687c22dccdd1ce3d789933" Jun 15 09:27:51.166804 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:51.166787 2562 scope.go:117] "RemoveContainer" containerID="58ccb9eae766ed1e79267218970231f43637920910687c22dccdd1ce3d789933" Jun 15 09:27:51.167088 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:27:51.167067 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58ccb9eae766ed1e79267218970231f43637920910687c22dccdd1ce3d789933\": container with ID starting with 58ccb9eae766ed1e79267218970231f43637920910687c22dccdd1ce3d789933 not found: ID does not exist" containerID="58ccb9eae766ed1e79267218970231f43637920910687c22dccdd1ce3d789933" Jun 15 09:27:51.167151 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:51.167097 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58ccb9eae766ed1e79267218970231f43637920910687c22dccdd1ce3d789933"} err="failed to get container status \"58ccb9eae766ed1e79267218970231f43637920910687c22dccdd1ce3d789933\": rpc error: code = NotFound desc = could not find container \"58ccb9eae766ed1e79267218970231f43637920910687c22dccdd1ce3d789933\": container with ID starting with 58ccb9eae766ed1e79267218970231f43637920910687c22dccdd1ce3d789933 not found: ID does not exist" Jun 15 09:27:51.178424 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:51.178402 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-776b74d8c5-v9fnj"] Jun 15 09:27:51.182188 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:51.182168 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-776b74d8c5-v9fnj"] Jun 15 09:27:51.935041 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:27:51.935004 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68493623-e470-46ff-ad9d-e5e51fde8162" path="/var/lib/kubelet/pods/68493623-e470-46ff-ad9d-e5e51fde8162/volumes" Jun 15 09:28:09.928813 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:09.928775 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:28:09.980346 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:09.980314 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-697554dff8-66j4v"] Jun 15 09:28:09.980619 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:09.980607 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68493623-e470-46ff-ad9d-e5e51fde8162" containerName="manager" Jun 15 09:28:09.980662 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:09.980620 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="68493623-e470-46ff-ad9d-e5e51fde8162" containerName="manager" Jun 15 09:28:09.980696 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:09.980682 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="68493623-e470-46ff-ad9d-e5e51fde8162" containerName="manager" Jun 15 09:28:09.983485 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:09.983468 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-697554dff8-66j4v" Jun 15 09:28:09.986022 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:09.985996 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-j7974\"" Jun 15 09:28:09.986143 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:09.986021 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 15 09:28:09.992267 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:09.992244 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-697554dff8-66j4v"] Jun 15 09:28:10.060653 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.060614 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk4xk\" (UniqueName: \"kubernetes.io/projected/93a0f85d-5bf7-4028-b63e-dfa26233f10c-kube-api-access-qk4xk\") pod \"maas-api-697554dff8-66j4v\" (UID: \"93a0f85d-5bf7-4028-b63e-dfa26233f10c\") " pod="opendatahub/maas-api-697554dff8-66j4v" Jun 15 09:28:10.060865 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.060708 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93a0f85d-5bf7-4028-b63e-dfa26233f10c-maas-api-tls\") pod \"maas-api-697554dff8-66j4v\" (UID: \"93a0f85d-5bf7-4028-b63e-dfa26233f10c\") " pod="opendatahub/maas-api-697554dff8-66j4v" Jun 15 09:28:10.060865 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.060752 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93a0f85d-5bf7-4028-b63e-dfa26233f10c-openshift-service-ca\") pod \"maas-api-697554dff8-66j4v\" (UID: \"93a0f85d-5bf7-4028-b63e-dfa26233f10c\") " pod="opendatahub/maas-api-697554dff8-66j4v" Jun 15 09:28:10.070798 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.070768 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:28:10.161178 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.161140 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93a0f85d-5bf7-4028-b63e-dfa26233f10c-openshift-service-ca\") pod \"maas-api-697554dff8-66j4v\" (UID: \"93a0f85d-5bf7-4028-b63e-dfa26233f10c\") " pod="opendatahub/maas-api-697554dff8-66j4v" Jun 15 09:28:10.161178 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.161183 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk4xk\" (UniqueName: \"kubernetes.io/projected/93a0f85d-5bf7-4028-b63e-dfa26233f10c-kube-api-access-qk4xk\") pod \"maas-api-697554dff8-66j4v\" (UID: \"93a0f85d-5bf7-4028-b63e-dfa26233f10c\") " pod="opendatahub/maas-api-697554dff8-66j4v" Jun 15 09:28:10.161404 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.161224 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93a0f85d-5bf7-4028-b63e-dfa26233f10c-maas-api-tls\") pod \"maas-api-697554dff8-66j4v\" (UID: \"93a0f85d-5bf7-4028-b63e-dfa26233f10c\") " pod="opendatahub/maas-api-697554dff8-66j4v" Jun 15 09:28:10.161812 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.161790 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93a0f85d-5bf7-4028-b63e-dfa26233f10c-openshift-service-ca\") pod \"maas-api-697554dff8-66j4v\" (UID: \"93a0f85d-5bf7-4028-b63e-dfa26233f10c\") " pod="opendatahub/maas-api-697554dff8-66j4v" Jun 15 09:28:10.163592 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.163563 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93a0f85d-5bf7-4028-b63e-dfa26233f10c-maas-api-tls\") pod \"maas-api-697554dff8-66j4v\" (UID: \"93a0f85d-5bf7-4028-b63e-dfa26233f10c\") " pod="opendatahub/maas-api-697554dff8-66j4v" Jun 15 09:28:10.168547 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.168524 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk4xk\" (UniqueName: \"kubernetes.io/projected/93a0f85d-5bf7-4028-b63e-dfa26233f10c-kube-api-access-qk4xk\") pod \"maas-api-697554dff8-66j4v\" (UID: \"93a0f85d-5bf7-4028-b63e-dfa26233f10c\") " pod="opendatahub/maas-api-697554dff8-66j4v" Jun 15 09:28:10.296897 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.296806 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-697554dff8-66j4v" Jun 15 09:28:10.326933 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.326896 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:28:10.414896 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.414826 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:28:10.415919 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.415898 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-hl8fp"] Jun 15 09:28:10.420645 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.420626 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-579785bc64-hl8fp" Jun 15 09:28:10.423079 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.423059 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-ns2pj\"" Jun 15 09:28:10.423169 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.423069 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 15 09:28:10.429584 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.429564 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-hl8fp"] Jun 15 09:28:10.435435 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.435417 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-697554dff8-66j4v"] Jun 15 09:28:10.438714 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:28:10.438692 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93a0f85d_5bf7_4028_b63e_dfa26233f10c.slice/crio-f56ba5eeae8258549ba8844d9bf5ec79b955afc80ab815cde64d50665e9d0d43 WatchSource:0}: Error finding container f56ba5eeae8258549ba8844d9bf5ec79b955afc80ab815cde64d50665e9d0d43: Status 404 returned error can't find the container with id f56ba5eeae8258549ba8844d9bf5ec79b955afc80ab815cde64d50665e9d0d43 Jun 15 09:28:10.463413 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.463377 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nck6t\" (UniqueName: \"kubernetes.io/projected/c80e6536-a697-4f6e-b3ab-b4f39b5d385d-kube-api-access-nck6t\") pod \"payload-pre-processing-579785bc64-hl8fp\" (UID: \"c80e6536-a697-4f6e-b3ab-b4f39b5d385d\") " pod="openshift-ingress/payload-pre-processing-579785bc64-hl8fp" Jun 15 09:28:10.476553 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.476520 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-fqnxb"] Jun 15 09:28:10.480051 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.480035 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-699b6cd6c-fqnxb" Jun 15 09:28:10.489115 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.489094 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-fqnxb"] Jun 15 09:28:10.564501 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.564419 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b784j\" (UniqueName: \"kubernetes.io/projected/4bb7231f-d2c5-421c-9f83-4254c4799157-kube-api-access-b784j\") pod \"payload-processing-699b6cd6c-fqnxb\" (UID: \"4bb7231f-d2c5-421c-9f83-4254c4799157\") " pod="openshift-ingress/payload-processing-699b6cd6c-fqnxb" Jun 15 09:28:10.564501 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.564468 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nck6t\" (UniqueName: \"kubernetes.io/projected/c80e6536-a697-4f6e-b3ab-b4f39b5d385d-kube-api-access-nck6t\") pod \"payload-pre-processing-579785bc64-hl8fp\" (UID: \"c80e6536-a697-4f6e-b3ab-b4f39b5d385d\") " pod="openshift-ingress/payload-pre-processing-579785bc64-hl8fp" Jun 15 09:28:10.572577 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.572549 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nck6t\" (UniqueName: \"kubernetes.io/projected/c80e6536-a697-4f6e-b3ab-b4f39b5d385d-kube-api-access-nck6t\") pod \"payload-pre-processing-579785bc64-hl8fp\" (UID: \"c80e6536-a697-4f6e-b3ab-b4f39b5d385d\") " pod="openshift-ingress/payload-pre-processing-579785bc64-hl8fp" Jun 15 09:28:10.665747 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.665715 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b784j\" (UniqueName: \"kubernetes.io/projected/4bb7231f-d2c5-421c-9f83-4254c4799157-kube-api-access-b784j\") pod \"payload-processing-699b6cd6c-fqnxb\" (UID: \"4bb7231f-d2c5-421c-9f83-4254c4799157\") " pod="openshift-ingress/payload-processing-699b6cd6c-fqnxb" Jun 15 09:28:10.673577 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.673549 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b784j\" (UniqueName: \"kubernetes.io/projected/4bb7231f-d2c5-421c-9f83-4254c4799157-kube-api-access-b784j\") pod \"payload-processing-699b6cd6c-fqnxb\" (UID: \"4bb7231f-d2c5-421c-9f83-4254c4799157\") " pod="openshift-ingress/payload-processing-699b6cd6c-fqnxb" Jun 15 09:28:10.732346 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.732311 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-579785bc64-hl8fp" Jun 15 09:28:10.792377 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.792158 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-699b6cd6c-fqnxb" Jun 15 09:28:10.874966 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.874931 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-hl8fp"] Jun 15 09:28:10.878767 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:28:10.878739 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc80e6536_a697_4f6e_b3ab_b4f39b5d385d.slice/crio-9b86233ccebc66b75a7565f4741827feb13d9581cbeb919f23f6aa76b5cd7e3a WatchSource:0}: Error finding container 9b86233ccebc66b75a7565f4741827feb13d9581cbeb919f23f6aa76b5cd7e3a: Status 404 returned error can't find the container with id 9b86233ccebc66b75a7565f4741827feb13d9581cbeb919f23f6aa76b5cd7e3a Jun 15 09:28:10.937448 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:10.937420 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-fqnxb"] Jun 15 09:28:10.940142 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:28:10.940108 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bb7231f_d2c5_421c_9f83_4254c4799157.slice/crio-c557fb116cd1e36725559fde855b5ea2c9b926af59269071ef3b56db3686f103 WatchSource:0}: Error finding container c557fb116cd1e36725559fde855b5ea2c9b926af59269071ef3b56db3686f103: Status 404 returned error can't find the container with id c557fb116cd1e36725559fde855b5ea2c9b926af59269071ef3b56db3686f103 Jun 15 09:28:11.256408 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:11.256315 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-697554dff8-66j4v" event={"ID":"93a0f85d-5bf7-4028-b63e-dfa26233f10c","Type":"ContainerStarted","Data":"f56ba5eeae8258549ba8844d9bf5ec79b955afc80ab815cde64d50665e9d0d43"} Jun 15 09:28:11.266702 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:11.266661 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-699b6cd6c-fqnxb" event={"ID":"4bb7231f-d2c5-421c-9f83-4254c4799157","Type":"ContainerStarted","Data":"c557fb116cd1e36725559fde855b5ea2c9b926af59269071ef3b56db3686f103"} Jun 15 09:28:11.268298 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:11.268261 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-579785bc64-hl8fp" event={"ID":"c80e6536-a697-4f6e-b3ab-b4f39b5d385d","Type":"ContainerStarted","Data":"9b86233ccebc66b75a7565f4741827feb13d9581cbeb919f23f6aa76b5cd7e3a"} Jun 15 09:28:14.284136 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:14.284104 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-579785bc64-hl8fp" event={"ID":"c80e6536-a697-4f6e-b3ab-b4f39b5d385d","Type":"ContainerStarted","Data":"6bcfaffdc53eba622b461febde26d4530287a038b405feef173029e6eec5fb94"} Jun 15 09:28:14.284546 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:14.284223 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-579785bc64-hl8fp" Jun 15 09:28:14.285810 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:14.285789 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-697554dff8-66j4v" event={"ID":"93a0f85d-5bf7-4028-b63e-dfa26233f10c","Type":"ContainerStarted","Data":"6cfb8959aeabfb285a7c1043a1a9aa724218cfeefc801583bbb3e3090e4f5892"} Jun 15 09:28:14.286207 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:14.286192 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-697554dff8-66j4v" Jun 15 09:28:14.287366 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:14.287347 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-699b6cd6c-fqnxb" event={"ID":"4bb7231f-d2c5-421c-9f83-4254c4799157","Type":"ContainerStarted","Data":"d73be788dfdee7b123105a06207a66fb0d1608074586986c233826433bdc5fff"} Jun 15 09:28:14.287682 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:14.287668 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-699b6cd6c-fqnxb" Jun 15 09:28:14.299525 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:14.299482 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-579785bc64-hl8fp" podStartSLOduration=1.080658748 podStartE2EDuration="4.299469782s" totalImagesPullingTime="3.218811034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:28:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:28:14.298370054 +0000 UTC m=+578.861792359" watchObservedRunningTime="2026-06-15 09:28:14.299469782 +0000 UTC m=+578.862892086" Jun 15 09:28:14.314286 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:14.314244 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-697554dff8-66j4v" podStartSLOduration=1.6856298079999998 podStartE2EDuration="5.314231503s" totalImagesPullingTime="3.628601695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:28:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:28:14.31302653 +0000 UTC m=+578.876448835" watchObservedRunningTime="2026-06-15 09:28:14.314231503 +0000 UTC m=+578.877653808" Jun 15 09:28:14.327545 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:14.327465 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-699b6cd6c-fqnxb" podStartSLOduration=1.166693333 podStartE2EDuration="4.327449657s" totalImagesPullingTime="3.160756324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:28:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:28:14.327407573 +0000 UTC m=+578.890829891" watchObservedRunningTime="2026-06-15 09:28:14.327449657 +0000 UTC m=+578.890871962" Jun 15 09:28:18.308578 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:18.308481 2562 generic.go:356] "Generic (PLEG): container finished" podID="30d60e63-17db-47e9-800c-2c5bd368678e" containerID="6cbe5ca5371e6a7184546156adc6ff05affa6d7d0e5c7d4bf3ad6a948c852d0b" exitCode=0 Jun 15 09:28:18.308578 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:18.308559 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" event={"ID":"30d60e63-17db-47e9-800c-2c5bd368678e","Type":"ContainerDied","Data":"6cbe5ca5371e6a7184546156adc6ff05affa6d7d0e5c7d4bf3ad6a948c852d0b"} Jun 15 09:28:18.309016 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:18.308981 2562 scope.go:117] "RemoveContainer" containerID="6cbe5ca5371e6a7184546156adc6ff05affa6d7d0e5c7d4bf3ad6a948c852d0b" Jun 15 09:28:18.309975 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:18.309945 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 15 09:28:18.310037 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:18.310013 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 15 09:28:18.310083 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:18.310046 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 15 09:28:19.314289 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:19.314260 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" event={"ID":"30d60e63-17db-47e9-800c-2c5bd368678e","Type":"ContainerStarted","Data":"648674dfde726215be17b8b95c5632eda27318c09bbc0299618afc00b0deb3ca"} Jun 15 09:28:19.976088 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:19.976049 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:28:19.976288 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:19.976262 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-8559cd5744-qxvtx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body= Jun 15 09:28:19.976334 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:19.976316 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" podUID="30d60e63-17db-47e9-800c-2c5bd368678e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" Jun 15 09:28:20.976486 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:20.976451 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-8559cd5744-qxvtx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body= Jun 15 09:28:20.976914 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:20.976518 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" podUID="30d60e63-17db-47e9-800c-2c5bd368678e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" Jun 15 09:28:21.302334 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:21.302261 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-697554dff8-66j4v" Jun 15 09:28:21.976869 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:21.976816 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-8559cd5744-qxvtx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body= Jun 15 09:28:21.977252 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:21.976891 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" podUID="30d60e63-17db-47e9-800c-2c5bd368678e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" Jun 15 09:28:22.481168 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:22.481129 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 15 09:28:22.481353 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:22.481328 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="eec632c6-82b7-44c9-b493-c1b36fb7a53f" containerName="keycloak" containerID="cri-o://8cd39a7c15b1c114bca29cf82b30e617982120842e3b82a1dc63190b00d35674" gracePeriod=30 Jun 15 09:28:22.976939 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:22.976895 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-8559cd5744-qxvtx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body= Jun 15 09:28:22.977318 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:22.976962 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" podUID="30d60e63-17db-47e9-800c-2c5bd368678e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" Jun 15 09:28:23.980913 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:23.980883 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:28:23.981312 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:23.981286 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:28:23.982013 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:23.981993 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-qxvtx" Jun 15 09:28:24.035036 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.035014 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 15 09:28:24.097678 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.097650 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t94nw\" (UniqueName: \"kubernetes.io/projected/eec632c6-82b7-44c9-b493-c1b36fb7a53f-kube-api-access-t94nw\") pod \"eec632c6-82b7-44c9-b493-c1b36fb7a53f\" (UID: \"eec632c6-82b7-44c9-b493-c1b36fb7a53f\") " Jun 15 09:28:24.100187 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.100144 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eec632c6-82b7-44c9-b493-c1b36fb7a53f-kube-api-access-t94nw" (OuterVolumeSpecName: "kube-api-access-t94nw") pod "eec632c6-82b7-44c9-b493-c1b36fb7a53f" (UID: "eec632c6-82b7-44c9-b493-c1b36fb7a53f"). InnerVolumeSpecName "kube-api-access-t94nw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 09:28:24.199357 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.199287 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t94nw\" (UniqueName: \"kubernetes.io/projected/eec632c6-82b7-44c9-b493-c1b36fb7a53f-kube-api-access-t94nw\") on node \"ip-10-0-129-143.ec2.internal\" DevicePath \"\"" Jun 15 09:28:24.335608 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.335577 2562 generic.go:356] "Generic (PLEG): container finished" podID="eec632c6-82b7-44c9-b493-c1b36fb7a53f" containerID="8cd39a7c15b1c114bca29cf82b30e617982120842e3b82a1dc63190b00d35674" exitCode=143 Jun 15 09:28:24.335745 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.335624 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 15 09:28:24.335745 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.335665 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"eec632c6-82b7-44c9-b493-c1b36fb7a53f","Type":"ContainerDied","Data":"8cd39a7c15b1c114bca29cf82b30e617982120842e3b82a1dc63190b00d35674"} Jun 15 09:28:24.335745 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.335709 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"eec632c6-82b7-44c9-b493-c1b36fb7a53f","Type":"ContainerDied","Data":"4dbaaa8d03b7f7a127a25f5da2c91d849607839870d7d13c6194ee57c7c7cb57"} Jun 15 09:28:24.335745 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.335730 2562 scope.go:117] "RemoveContainer" containerID="8cd39a7c15b1c114bca29cf82b30e617982120842e3b82a1dc63190b00d35674" Jun 15 09:28:24.345009 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.344990 2562 scope.go:117] "RemoveContainer" containerID="8cd39a7c15b1c114bca29cf82b30e617982120842e3b82a1dc63190b00d35674" Jun 15 09:28:24.345259 ip-10-0-129-143 kubenswrapper[2562]: E0615 09:28:24.345241 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cd39a7c15b1c114bca29cf82b30e617982120842e3b82a1dc63190b00d35674\": container with ID starting with 8cd39a7c15b1c114bca29cf82b30e617982120842e3b82a1dc63190b00d35674 not found: ID does not exist" containerID="8cd39a7c15b1c114bca29cf82b30e617982120842e3b82a1dc63190b00d35674" Jun 15 09:28:24.345315 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.345266 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cd39a7c15b1c114bca29cf82b30e617982120842e3b82a1dc63190b00d35674"} err="failed to get container status \"8cd39a7c15b1c114bca29cf82b30e617982120842e3b82a1dc63190b00d35674\": rpc error: code = NotFound desc = could not find container \"8cd39a7c15b1c114bca29cf82b30e617982120842e3b82a1dc63190b00d35674\": container with ID starting with 8cd39a7c15b1c114bca29cf82b30e617982120842e3b82a1dc63190b00d35674 not found: ID does not exist" Jun 15 09:28:24.355606 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.355585 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 15 09:28:24.359236 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.359216 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 15 09:28:24.379016 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.378994 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 15 09:28:24.379344 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.379328 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eec632c6-82b7-44c9-b493-c1b36fb7a53f" containerName="keycloak" Jun 15 09:28:24.379413 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.379346 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="eec632c6-82b7-44c9-b493-c1b36fb7a53f" containerName="keycloak" Jun 15 09:28:24.379465 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.379420 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="eec632c6-82b7-44c9-b493-c1b36fb7a53f" containerName="keycloak" Jun 15 09:28:24.383263 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.383246 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 15 09:28:24.385618 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.385601 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 15 09:28:24.385714 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.385620 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 15 09:28:24.385776 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.385725 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-mmbq9\"" Jun 15 09:28:24.385776 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.385729 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 15 09:28:24.385903 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.385879 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 15 09:28:24.393104 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.393086 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 15 09:28:24.500877 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.500796 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp8bv\" (UniqueName: \"kubernetes.io/projected/031455ad-a220-491f-878f-30ae613d6f70-kube-api-access-bp8bv\") pod \"maas-keycloak-0\" (UID: \"031455ad-a220-491f-878f-30ae613d6f70\") " pod="keycloak-system/maas-keycloak-0" Jun 15 09:28:24.500877 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.500852 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/031455ad-a220-491f-878f-30ae613d6f70-test-realms\") pod \"maas-keycloak-0\" (UID: \"031455ad-a220-491f-878f-30ae613d6f70\") " pod="keycloak-system/maas-keycloak-0" Jun 15 09:28:24.601499 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.601474 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/031455ad-a220-491f-878f-30ae613d6f70-test-realms\") pod \"maas-keycloak-0\" (UID: \"031455ad-a220-491f-878f-30ae613d6f70\") " pod="keycloak-system/maas-keycloak-0" Jun 15 09:28:24.601617 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.601552 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp8bv\" (UniqueName: \"kubernetes.io/projected/031455ad-a220-491f-878f-30ae613d6f70-kube-api-access-bp8bv\") pod \"maas-keycloak-0\" (UID: \"031455ad-a220-491f-878f-30ae613d6f70\") " pod="keycloak-system/maas-keycloak-0" Jun 15 09:28:24.602268 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.602251 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/031455ad-a220-491f-878f-30ae613d6f70-test-realms\") pod \"maas-keycloak-0\" (UID: \"031455ad-a220-491f-878f-30ae613d6f70\") " pod="keycloak-system/maas-keycloak-0" Jun 15 09:28:24.609226 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.609209 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp8bv\" (UniqueName: \"kubernetes.io/projected/031455ad-a220-491f-878f-30ae613d6f70-kube-api-access-bp8bv\") pod \"maas-keycloak-0\" (UID: \"031455ad-a220-491f-878f-30ae613d6f70\") " pod="keycloak-system/maas-keycloak-0" Jun 15 09:28:24.693288 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.693256 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 15 09:28:24.817120 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:24.817098 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 15 09:28:24.819186 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:28:24.819163 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod031455ad_a220_491f_878f_30ae613d6f70.slice/crio-f187daca1f9a217f2dbcc39d1089f8318a725e1fe2b978bba50c21b655e3cfb2 WatchSource:0}: Error finding container f187daca1f9a217f2dbcc39d1089f8318a725e1fe2b978bba50c21b655e3cfb2: Status 404 returned error can't find the container with id f187daca1f9a217f2dbcc39d1089f8318a725e1fe2b978bba50c21b655e3cfb2 Jun 15 09:28:25.294520 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:25.294496 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-579785bc64-hl8fp" Jun 15 09:28:25.343214 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:25.343182 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"031455ad-a220-491f-878f-30ae613d6f70","Type":"ContainerStarted","Data":"90757f858b0ef16ccd7bf732b9320593a7d9a86005986e1afe888e4172097611"} Jun 15 09:28:25.343413 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:25.343242 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"031455ad-a220-491f-878f-30ae613d6f70","Type":"ContainerStarted","Data":"f187daca1f9a217f2dbcc39d1089f8318a725e1fe2b978bba50c21b655e3cfb2"} Jun 15 09:28:25.359751 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:25.359701 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.042994712 podStartE2EDuration="1.359683183s" totalImagesPullingTime="316.688471ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:28:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:28:25.35876706 +0000 UTC m=+589.922189366" watchObservedRunningTime="2026-06-15 09:28:25.359683183 +0000 UTC m=+589.923105526" Jun 15 09:28:25.693888 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:25.693824 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 15 09:28:25.695598 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:25.695564 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="031455ad-a220-491f-878f-30ae613d6f70" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.32:9000/health/started\": dial tcp 10.133.0.32:9000: connect: connection refused" Jun 15 09:28:25.938428 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:25.938385 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eec632c6-82b7-44c9-b493-c1b36fb7a53f" path="/var/lib/kubelet/pods/eec632c6-82b7-44c9-b493-c1b36fb7a53f/volumes" Jun 15 09:28:26.299014 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:26.298984 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-699b6cd6c-fqnxb" Jun 15 09:28:26.694380 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:26.694332 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="031455ad-a220-491f-878f-30ae613d6f70" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.32:9000/health/started\": dial tcp 10.133.0.32:9000: connect: connection refused" Jun 15 09:28:27.694028 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:27.693979 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="031455ad-a220-491f-878f-30ae613d6f70" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.32:9000/health/started\": dial tcp 10.133.0.32:9000: connect: connection refused" Jun 15 09:28:28.694805 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:28.694753 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="031455ad-a220-491f-878f-30ae613d6f70" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.32:9000/health/started\": dial tcp 10.133.0.32:9000: connect: connection refused" Jun 15 09:28:29.694428 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:29.694366 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="031455ad-a220-491f-878f-30ae613d6f70" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.32:9000/health/started\": dial tcp 10.133.0.32:9000: connect: connection refused" Jun 15 09:28:30.694337 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:30.694282 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="031455ad-a220-491f-878f-30ae613d6f70" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.32:9000/health/started\": dial tcp 10.133.0.32:9000: connect: connection refused" Jun 15 09:28:31.694356 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:31.694311 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="031455ad-a220-491f-878f-30ae613d6f70" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.32:9000/health/started\": dial tcp 10.133.0.32:9000: connect: connection refused" Jun 15 09:28:32.694275 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:32.694229 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="031455ad-a220-491f-878f-30ae613d6f70" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.32:9000/health/started\": dial tcp 10.133.0.32:9000: connect: connection refused" Jun 15 09:28:33.694668 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:33.694621 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="031455ad-a220-491f-878f-30ae613d6f70" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.32:9000/health/started\": dial tcp 10.133.0.32:9000: connect: connection refused" Jun 15 09:28:34.694104 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:34.694054 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 15 09:28:34.694414 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:34.694383 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="031455ad-a220-491f-878f-30ae613d6f70" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.32:9000/health/started\": dial tcp 10.133.0.32:9000: connect: connection refused" Jun 15 09:28:35.341192 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:35.341148 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:28:35.694401 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:35.694294 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="031455ad-a220-491f-878f-30ae613d6f70" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.32:9000/health/started\": dial tcp 10.133.0.32:9000: connect: connection refused" Jun 15 09:28:35.903032 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:35.902995 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pxdxj_f344540f-2677-49dd-8a0f-106be00f414d/ovn-acl-logging/0.log" Jun 15 09:28:35.903455 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:35.903435 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pxdxj_f344540f-2677-49dd-8a0f-106be00f414d/ovn-acl-logging/0.log" Jun 15 09:28:36.693769 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:36.693714 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="031455ad-a220-491f-878f-30ae613d6f70" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.32:9000/health/started\": dial tcp 10.133.0.32:9000: connect: connection refused" Jun 15 09:28:37.694316 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:37.694253 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="031455ad-a220-491f-878f-30ae613d6f70" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.32:9000/health/started\": dial tcp 10.133.0.32:9000: connect: connection refused" Jun 15 09:28:38.819063 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:38.817717 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 15 09:28:38.837608 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:38.837558 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="031455ad-a220-491f-878f-30ae613d6f70" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 15 09:28:46.169255 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:46.169213 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:28:47.197638 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:47.197596 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:28:48.824966 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:28:48.824932 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 15 09:29:01.910046 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:01.909961 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:29:02.004725 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:02.004693 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:29:12.115521 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.115461 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq"] Jun 15 09:29:12.118991 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.118970 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:12.122564 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.122542 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 15 09:29:12.122564 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.122556 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vdshl\"" Jun 15 09:29:12.122740 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.122618 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 15 09:29:12.122740 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.122544 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 15 09:29:12.130208 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.130188 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq"] Jun 15 09:29:12.160368 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.160340 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:29:12.203660 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.203636 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bt47\" (UniqueName: \"kubernetes.io/projected/67d674ce-c275-4b09-8975-4d7d58eb9de3-kube-api-access-4bt47\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq\" (UID: \"67d674ce-c275-4b09-8975-4d7d58eb9de3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:12.203781 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.203671 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67d674ce-c275-4b09-8975-4d7d58eb9de3-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq\" (UID: \"67d674ce-c275-4b09-8975-4d7d58eb9de3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:12.203781 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.203689 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67d674ce-c275-4b09-8975-4d7d58eb9de3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq\" (UID: \"67d674ce-c275-4b09-8975-4d7d58eb9de3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:12.203933 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.203791 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67d674ce-c275-4b09-8975-4d7d58eb9de3-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq\" (UID: \"67d674ce-c275-4b09-8975-4d7d58eb9de3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:12.203933 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.203851 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67d674ce-c275-4b09-8975-4d7d58eb9de3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq\" (UID: \"67d674ce-c275-4b09-8975-4d7d58eb9de3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:12.203933 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.203881 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/67d674ce-c275-4b09-8975-4d7d58eb9de3-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq\" (UID: \"67d674ce-c275-4b09-8975-4d7d58eb9de3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:12.304778 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.304750 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bt47\" (UniqueName: \"kubernetes.io/projected/67d674ce-c275-4b09-8975-4d7d58eb9de3-kube-api-access-4bt47\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq\" (UID: \"67d674ce-c275-4b09-8975-4d7d58eb9de3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:12.305031 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.304796 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67d674ce-c275-4b09-8975-4d7d58eb9de3-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq\" (UID: \"67d674ce-c275-4b09-8975-4d7d58eb9de3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:12.305031 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.304823 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67d674ce-c275-4b09-8975-4d7d58eb9de3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq\" (UID: \"67d674ce-c275-4b09-8975-4d7d58eb9de3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:12.305031 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.304898 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67d674ce-c275-4b09-8975-4d7d58eb9de3-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq\" (UID: \"67d674ce-c275-4b09-8975-4d7d58eb9de3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:12.305031 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.304932 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67d674ce-c275-4b09-8975-4d7d58eb9de3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq\" (UID: \"67d674ce-c275-4b09-8975-4d7d58eb9de3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:12.305031 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.304957 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/67d674ce-c275-4b09-8975-4d7d58eb9de3-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq\" (UID: \"67d674ce-c275-4b09-8975-4d7d58eb9de3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:12.305344 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.305214 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67d674ce-c275-4b09-8975-4d7d58eb9de3-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq\" (UID: \"67d674ce-c275-4b09-8975-4d7d58eb9de3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:12.305344 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.305273 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67d674ce-c275-4b09-8975-4d7d58eb9de3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq\" (UID: \"67d674ce-c275-4b09-8975-4d7d58eb9de3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:12.305444 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.305376 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/67d674ce-c275-4b09-8975-4d7d58eb9de3-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq\" (UID: \"67d674ce-c275-4b09-8975-4d7d58eb9de3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:12.306950 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.306934 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67d674ce-c275-4b09-8975-4d7d58eb9de3-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq\" (UID: \"67d674ce-c275-4b09-8975-4d7d58eb9de3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:12.307365 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.307351 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67d674ce-c275-4b09-8975-4d7d58eb9de3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq\" (UID: \"67d674ce-c275-4b09-8975-4d7d58eb9de3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:12.313423 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.313403 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bt47\" (UniqueName: \"kubernetes.io/projected/67d674ce-c275-4b09-8975-4d7d58eb9de3-kube-api-access-4bt47\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq\" (UID: \"67d674ce-c275-4b09-8975-4d7d58eb9de3\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:12.429933 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.429826 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:12.559462 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.559438 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq"] Jun 15 09:29:12.561655 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:29:12.561619 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67d674ce_c275_4b09_8975_4d7d58eb9de3.slice/crio-7cc009d72be2dad42083c49c329b52d1d3e936dfbb5673f5275a0a0e6135ebdf WatchSource:0}: Error finding container 7cc009d72be2dad42083c49c329b52d1d3e936dfbb5673f5275a0a0e6135ebdf: Status 404 returned error can't find the container with id 7cc009d72be2dad42083c49c329b52d1d3e936dfbb5673f5275a0a0e6135ebdf Jun 15 09:29:12.567720 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.567699 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 15 09:29:12.594447 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:12.594421 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" event={"ID":"67d674ce-c275-4b09-8975-4d7d58eb9de3","Type":"ContainerStarted","Data":"7cc009d72be2dad42083c49c329b52d1d3e936dfbb5673f5275a0a0e6135ebdf"} Jun 15 09:29:16.615073 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:16.615025 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" event={"ID":"67d674ce-c275-4b09-8975-4d7d58eb9de3","Type":"ContainerStarted","Data":"22a3dd732fbae6e442dc7f484e12c4a3b24ce88e66a0671fa9d00f1c349221cf"} Jun 15 09:29:22.867737 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:22.867700 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:29:34.544392 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:34.544344 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:29:45.741534 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:45.741503 2562 generic.go:356] "Generic (PLEG): container finished" podID="67d674ce-c275-4b09-8975-4d7d58eb9de3" containerID="22a3dd732fbae6e442dc7f484e12c4a3b24ce88e66a0671fa9d00f1c349221cf" exitCode=0 Jun 15 09:29:45.741954 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:45.741580 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" event={"ID":"67d674ce-c275-4b09-8975-4d7d58eb9de3","Type":"ContainerDied","Data":"22a3dd732fbae6e442dc7f484e12c4a3b24ce88e66a0671fa9d00f1c349221cf"} Jun 15 09:29:47.754123 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:47.754089 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" event={"ID":"67d674ce-c275-4b09-8975-4d7d58eb9de3","Type":"ContainerStarted","Data":"bd0687fb3c3ce6ce372dbb175e01b907d2ad31f36f510109826c25948d83e72a"} Jun 15 09:29:47.771483 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:47.771438 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" podStartSLOduration=2.572119519 podStartE2EDuration="35.771421399s" totalImagesPullingTime="4.17506202s" totalInitContainerRuntime="29.02423986s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:29:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:29:47.771279248 +0000 UTC m=+672.334701555" watchObservedRunningTime="2026-06-15 09:29:47.771421399 +0000 UTC m=+672.334843707" Jun 15 09:29:52.430390 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:52.430351 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:52.430808 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:52.430408 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:52.442480 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:52.442457 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:52.786987 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:52.786892 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq" Jun 15 09:29:55.929137 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:55.929101 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr"] Jun 15 09:29:55.932259 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:55.932238 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:29:55.934692 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:55.934669 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 15 09:29:55.943118 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:55.943098 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr"] Jun 15 09:29:55.963993 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:55.963970 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:29:55.973013 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:55.972993 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd86450b-e63d-4707-a33b-1755cb61fd17-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr\" (UID: \"bd86450b-e63d-4707-a33b-1755cb61fd17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:29:55.973101 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:55.973057 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bd86450b-e63d-4707-a33b-1755cb61fd17-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr\" (UID: \"bd86450b-e63d-4707-a33b-1755cb61fd17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:29:55.973101 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:55.973078 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd86450b-e63d-4707-a33b-1755cb61fd17-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr\" (UID: \"bd86450b-e63d-4707-a33b-1755cb61fd17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:29:55.973101 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:55.973093 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd86450b-e63d-4707-a33b-1755cb61fd17-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr\" (UID: \"bd86450b-e63d-4707-a33b-1755cb61fd17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:29:55.973244 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:55.973163 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd86450b-e63d-4707-a33b-1755cb61fd17-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr\" (UID: \"bd86450b-e63d-4707-a33b-1755cb61fd17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:29:55.973244 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:55.973206 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp9jc\" (UniqueName: \"kubernetes.io/projected/bd86450b-e63d-4707-a33b-1755cb61fd17-kube-api-access-xp9jc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr\" (UID: \"bd86450b-e63d-4707-a33b-1755cb61fd17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:29:56.073652 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:56.073623 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bd86450b-e63d-4707-a33b-1755cb61fd17-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr\" (UID: \"bd86450b-e63d-4707-a33b-1755cb61fd17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:29:56.073831 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:56.073663 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd86450b-e63d-4707-a33b-1755cb61fd17-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr\" (UID: \"bd86450b-e63d-4707-a33b-1755cb61fd17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:29:56.073831 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:56.073680 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd86450b-e63d-4707-a33b-1755cb61fd17-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr\" (UID: \"bd86450b-e63d-4707-a33b-1755cb61fd17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:29:56.073831 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:56.073717 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd86450b-e63d-4707-a33b-1755cb61fd17-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr\" (UID: \"bd86450b-e63d-4707-a33b-1755cb61fd17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:29:56.073831 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:56.073753 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp9jc\" (UniqueName: \"kubernetes.io/projected/bd86450b-e63d-4707-a33b-1755cb61fd17-kube-api-access-xp9jc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr\" (UID: \"bd86450b-e63d-4707-a33b-1755cb61fd17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:29:56.073831 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:56.073810 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd86450b-e63d-4707-a33b-1755cb61fd17-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr\" (UID: \"bd86450b-e63d-4707-a33b-1755cb61fd17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:29:56.074238 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:56.074182 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd86450b-e63d-4707-a33b-1755cb61fd17-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr\" (UID: \"bd86450b-e63d-4707-a33b-1755cb61fd17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:29:56.074342 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:56.074274 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd86450b-e63d-4707-a33b-1755cb61fd17-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr\" (UID: \"bd86450b-e63d-4707-a33b-1755cb61fd17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:29:56.074342 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:56.074322 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bd86450b-e63d-4707-a33b-1755cb61fd17-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr\" (UID: \"bd86450b-e63d-4707-a33b-1755cb61fd17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:29:56.076195 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:56.076170 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd86450b-e63d-4707-a33b-1755cb61fd17-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr\" (UID: \"bd86450b-e63d-4707-a33b-1755cb61fd17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:29:56.076404 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:56.076383 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd86450b-e63d-4707-a33b-1755cb61fd17-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr\" (UID: \"bd86450b-e63d-4707-a33b-1755cb61fd17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:29:56.081315 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:56.081293 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp9jc\" (UniqueName: \"kubernetes.io/projected/bd86450b-e63d-4707-a33b-1755cb61fd17-kube-api-access-xp9jc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr\" (UID: \"bd86450b-e63d-4707-a33b-1755cb61fd17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:29:56.242694 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:56.242624 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:29:56.368384 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:56.368358 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr"] Jun 15 09:29:56.370548 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:29:56.370517 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd86450b_e63d_4707_a33b_1755cb61fd17.slice/crio-c5ad1c869c4da8b1f088e836d10e1c1f9bca70909838b55f03b6cfe61e2d5e71 WatchSource:0}: Error finding container c5ad1c869c4da8b1f088e836d10e1c1f9bca70909838b55f03b6cfe61e2d5e71: Status 404 returned error can't find the container with id c5ad1c869c4da8b1f088e836d10e1c1f9bca70909838b55f03b6cfe61e2d5e71 Jun 15 09:29:56.795819 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:56.795787 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" event={"ID":"bd86450b-e63d-4707-a33b-1755cb61fd17","Type":"ContainerStarted","Data":"e6ba689d96988da66335166249029707935612b56f8cb025dd9d866cce4e0398"} Jun 15 09:29:56.795998 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:56.795855 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" event={"ID":"bd86450b-e63d-4707-a33b-1755cb61fd17","Type":"ContainerStarted","Data":"c5ad1c869c4da8b1f088e836d10e1c1f9bca70909838b55f03b6cfe61e2d5e71"} Jun 15 09:29:59.811314 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:59.811282 2562 generic.go:356] "Generic (PLEG): container finished" podID="bd86450b-e63d-4707-a33b-1755cb61fd17" containerID="e6ba689d96988da66335166249029707935612b56f8cb025dd9d866cce4e0398" exitCode=0 Jun 15 09:29:59.811694 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:29:59.811325 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" event={"ID":"bd86450b-e63d-4707-a33b-1755cb61fd17","Type":"ContainerDied","Data":"e6ba689d96988da66335166249029707935612b56f8cb025dd9d866cce4e0398"} Jun 15 09:30:00.817085 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:00.817053 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" event={"ID":"bd86450b-e63d-4707-a33b-1755cb61fd17","Type":"ContainerStarted","Data":"23232d3b8dbe077bd8350bfae5c0ec11ec9fe0ce0d9f6abaeeb97aca4cca968d"} Jun 15 09:30:00.842164 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:00.842119 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" podStartSLOduration=2.342020674 podStartE2EDuration="5.842103842s" totalImagesPullingTime="269.290085ms" totalInitContainerRuntime="3.230793083s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:29:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:30:00.840032234 +0000 UTC m=+685.403454625" watchObservedRunningTime="2026-06-15 09:30:00.842103842 +0000 UTC m=+685.405526214" Jun 15 09:30:06.243405 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:06.243373 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:30:06.243828 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:06.243422 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:30:06.255362 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:06.255337 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:30:06.854269 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:06.854240 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr" Jun 15 09:30:17.043629 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:17.043598 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:30:21.818760 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:21.818730 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr"] Jun 15 09:30:21.824872 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:21.824829 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:21.827301 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:21.827276 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jun 15 09:30:21.833894 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:21.833870 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr"] Jun 15 09:30:21.849306 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:21.849284 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:30:21.895109 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:21.895082 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a93dd1e4-5736-48c1-9c18-cd8f938ae791-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kdmxr\" (UID: \"a93dd1e4-5736-48c1-9c18-cd8f938ae791\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:21.895236 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:21.895115 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j79tp\" (UniqueName: \"kubernetes.io/projected/a93dd1e4-5736-48c1-9c18-cd8f938ae791-kube-api-access-j79tp\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kdmxr\" (UID: \"a93dd1e4-5736-48c1-9c18-cd8f938ae791\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:21.895236 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:21.895140 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a93dd1e4-5736-48c1-9c18-cd8f938ae791-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kdmxr\" (UID: \"a93dd1e4-5736-48c1-9c18-cd8f938ae791\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:21.895236 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:21.895162 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a93dd1e4-5736-48c1-9c18-cd8f938ae791-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kdmxr\" (UID: \"a93dd1e4-5736-48c1-9c18-cd8f938ae791\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:21.895345 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:21.895243 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a93dd1e4-5736-48c1-9c18-cd8f938ae791-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kdmxr\" (UID: \"a93dd1e4-5736-48c1-9c18-cd8f938ae791\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:21.895345 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:21.895271 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a93dd1e4-5736-48c1-9c18-cd8f938ae791-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kdmxr\" (UID: \"a93dd1e4-5736-48c1-9c18-cd8f938ae791\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:21.995997 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:21.995954 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a93dd1e4-5736-48c1-9c18-cd8f938ae791-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kdmxr\" (UID: \"a93dd1e4-5736-48c1-9c18-cd8f938ae791\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:21.995997 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:21.995999 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j79tp\" (UniqueName: \"kubernetes.io/projected/a93dd1e4-5736-48c1-9c18-cd8f938ae791-kube-api-access-j79tp\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kdmxr\" (UID: \"a93dd1e4-5736-48c1-9c18-cd8f938ae791\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:21.996217 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:21.996027 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a93dd1e4-5736-48c1-9c18-cd8f938ae791-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kdmxr\" (UID: \"a93dd1e4-5736-48c1-9c18-cd8f938ae791\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:21.996217 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:21.996058 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a93dd1e4-5736-48c1-9c18-cd8f938ae791-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kdmxr\" (UID: \"a93dd1e4-5736-48c1-9c18-cd8f938ae791\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:21.996217 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:21.996118 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a93dd1e4-5736-48c1-9c18-cd8f938ae791-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kdmxr\" (UID: \"a93dd1e4-5736-48c1-9c18-cd8f938ae791\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:21.996217 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:21.996145 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a93dd1e4-5736-48c1-9c18-cd8f938ae791-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kdmxr\" (UID: \"a93dd1e4-5736-48c1-9c18-cd8f938ae791\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:21.996510 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:21.996481 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a93dd1e4-5736-48c1-9c18-cd8f938ae791-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kdmxr\" (UID: \"a93dd1e4-5736-48c1-9c18-cd8f938ae791\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:21.996622 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:21.996511 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a93dd1e4-5736-48c1-9c18-cd8f938ae791-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kdmxr\" (UID: \"a93dd1e4-5736-48c1-9c18-cd8f938ae791\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:21.996622 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:21.996536 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a93dd1e4-5736-48c1-9c18-cd8f938ae791-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kdmxr\" (UID: \"a93dd1e4-5736-48c1-9c18-cd8f938ae791\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:21.998236 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:21.998205 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a93dd1e4-5736-48c1-9c18-cd8f938ae791-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kdmxr\" (UID: \"a93dd1e4-5736-48c1-9c18-cd8f938ae791\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:21.998674 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:21.998658 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a93dd1e4-5736-48c1-9c18-cd8f938ae791-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kdmxr\" (UID: \"a93dd1e4-5736-48c1-9c18-cd8f938ae791\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:22.003469 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:22.003446 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j79tp\" (UniqueName: \"kubernetes.io/projected/a93dd1e4-5736-48c1-9c18-cd8f938ae791-kube-api-access-j79tp\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kdmxr\" (UID: \"a93dd1e4-5736-48c1-9c18-cd8f938ae791\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:22.135131 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:22.135031 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:22.265509 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:22.265484 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr"] Jun 15 09:30:22.267561 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:30:22.267533 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda93dd1e4_5736_48c1_9c18_cd8f938ae791.slice/crio-d1eb12169c0268009a9cb75123bdd00786c02b4011a2b8e24dbb40c33af11634 WatchSource:0}: Error finding container d1eb12169c0268009a9cb75123bdd00786c02b4011a2b8e24dbb40c33af11634: Status 404 returned error can't find the container with id d1eb12169c0268009a9cb75123bdd00786c02b4011a2b8e24dbb40c33af11634 Jun 15 09:30:22.912741 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:22.912712 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" event={"ID":"a93dd1e4-5736-48c1-9c18-cd8f938ae791","Type":"ContainerStarted","Data":"fb9132c86cb168a48e2b43fd11bd725942d2735fe84cb781ba710937f711c31c"} Jun 15 09:30:22.913118 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:22.912750 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" event={"ID":"a93dd1e4-5736-48c1-9c18-cd8f938ae791","Type":"ContainerStarted","Data":"d1eb12169c0268009a9cb75123bdd00786c02b4011a2b8e24dbb40c33af11634"} Jun 15 09:30:23.918604 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:23.918522 2562 generic.go:356] "Generic (PLEG): container finished" podID="a93dd1e4-5736-48c1-9c18-cd8f938ae791" containerID="fb9132c86cb168a48e2b43fd11bd725942d2735fe84cb781ba710937f711c31c" exitCode=0 Jun 15 09:30:23.918966 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:23.918612 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" event={"ID":"a93dd1e4-5736-48c1-9c18-cd8f938ae791","Type":"ContainerDied","Data":"fb9132c86cb168a48e2b43fd11bd725942d2735fe84cb781ba710937f711c31c"} Jun 15 09:30:24.924261 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:24.924220 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" event={"ID":"a93dd1e4-5736-48c1-9c18-cd8f938ae791","Type":"ContainerStarted","Data":"5fcb3888fdf2213021eafddc1ede81ab6715e8457d5bf5384e6c50650db23c32"} Jun 15 09:30:24.943093 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:24.943049 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" podStartSLOduration=2.436986785 podStartE2EDuration="3.943033591s" totalImagesPullingTime="226.603043ms" totalInitContainerRuntime="1.279443763s" isStatefulPod=false podCreationTimestamp="2026-06-15 09:30:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:30:24.941547462 +0000 UTC m=+709.504969766" watchObservedRunningTime="2026-06-15 09:30:24.943033591 +0000 UTC m=+709.506455896" Jun 15 09:30:25.905322 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:25.905286 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:30:25.957656 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:25.957612 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:30:30.058440 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:30.058405 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:30:32.135402 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:32.135370 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:32.135759 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:32.135414 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:32.147859 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:32.147817 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:32.969480 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:32.969455 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kdmxr" Jun 15 09:30:36.150311 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:36.150277 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:30:36.252033 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:36.252003 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:30:45.502732 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:30:45.502680 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:33:35.935987 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:33:35.935913 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pxdxj_f344540f-2677-49dd-8a0f-106be00f414d/ovn-acl-logging/0.log" Jun 15 09:33:35.937889 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:33:35.937866 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pxdxj_f344540f-2677-49dd-8a0f-106be00f414d/ovn-acl-logging/0.log" Jun 15 09:34:10.748348 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:10.748317 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-697554dff8-66j4v_93a0f85d-5bf7-4028-b63e-dfa26233f10c/maas-api/0.log" Jun 15 09:34:10.974232 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:10.974203 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-767f9856ff-sxbgt_85021cfa-1559-4ece-81f3-556fe5004e5a/manager/0.log" Jun 15 09:34:11.449581 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:11.449549 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5659c68bbf-jjfvb_aafe0dfe-75d7-42b6-979c-a466e4edf40b/copy-manifests/0.log" Jun 15 09:34:11.455672 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:11.455654 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5659c68bbf-jjfvb_aafe0dfe-75d7-42b6-979c-a466e4edf40b/manager/0.log" Jun 15 09:34:11.687784 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:11.687755 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-6p2lm_6f42c62e-7931-4fb9-857f-b334967d9e9a/postgres/0.log" Jun 15 09:34:13.273242 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:13.273206 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-nzss9_9fa3a695-3004-4447-bc80-c3a5b102cf71/kuadrant-console-plugin/0.log" Jun 15 09:34:14.372501 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:14.372465 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bcbb8b488-cr726_28a85a02-7443-41b3-a429-ef011dc7f3c2/kube-auth-proxy/0.log" Jun 15 09:34:14.485145 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:14.485113 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-qxvtx_30d60e63-17db-47e9-800c-2c5bd368678e/istio-proxy/1.log" Jun 15 09:34:14.592233 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:14.592201 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-hl8fp_c80e6536-a697-4f6e-b3ab-b4f39b5d385d/payload-pre-processing/0.log" Jun 15 09:34:14.706077 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:14.706002 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-fqnxb_4bb7231f-d2c5-421c-9f83-4254c4799157/payload-processing/0.log" Jun 15 09:34:15.174072 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:15.174046 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq_67d674ce-c275-4b09-8975-4d7d58eb9de3/storage-initializer/0.log" Jun 15 09:34:15.181019 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:15.180992 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-cfdcq_67d674ce-c275-4b09-8975-4d7d58eb9de3/main/0.log" Jun 15 09:34:15.524950 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:15.524872 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr_bd86450b-e63d-4707-a33b-1755cb61fd17/storage-initializer/0.log" Jun 15 09:34:15.531961 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:15.531936 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cbpkwr_bd86450b-e63d-4707-a33b-1755cb61fd17/main/0.log" Jun 15 09:34:15.751982 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:15.751954 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-kdmxr_a93dd1e4-5736-48c1-9c18-cd8f938ae791/storage-initializer/0.log" Jun 15 09:34:15.758248 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:15.758224 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-kdmxr_a93dd1e4-5736-48c1-9c18-cd8f938ae791/main/0.log" Jun 15 09:34:20.057461 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:20.057431 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:34:21.183617 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:21.183587 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" limit=1024 Jun 15 09:34:22.718780 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:22.718752 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kddf5_56a0c215-5d6d-4cb2-b9aa-505333bc0cea/global-pull-secret-syncer/0.log" Jun 15 09:34:22.848065 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:22.848040 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-47q6n_dd2e6f0c-a1ba-4f06-8286-ff69fdf52440/konnectivity-agent/0.log" Jun 15 09:34:22.921303 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:22.921277 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-143.ec2.internal_6546904cab304243cc28b9f8f6f2abcd/haproxy/0.log" Jun 15 09:34:28.270445 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:28.270427 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-nzss9_9fa3a695-3004-4447-bc80-c3a5b102cf71/kuadrant-console-plugin/0.log" Jun 15 09:34:30.378297 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:30.378270 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sv9zr_9f44c5b9-3847-4f50-bc6f-809b7569c288/node-exporter/0.log" Jun 15 09:34:30.401151 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:30.401129 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sv9zr_9f44c5b9-3847-4f50-bc6f-809b7569c288/kube-rbac-proxy/0.log" Jun 15 09:34:30.422826 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:30.422803 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sv9zr_9f44c5b9-3847-4f50-bc6f-809b7569c288/init-textfile/0.log" Jun 15 09:34:30.633552 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:30.633479 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_38d75778-28a1-4c29-bcb2-46470d15cae4/prometheus/0.log" Jun 15 09:34:30.654323 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:30.654302 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_38d75778-28a1-4c29-bcb2-46470d15cae4/config-reloader/0.log" Jun 15 09:34:30.674037 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:30.674013 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_38d75778-28a1-4c29-bcb2-46470d15cae4/thanos-sidecar/0.log" Jun 15 09:34:30.697141 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:30.697119 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_38d75778-28a1-4c29-bcb2-46470d15cae4/kube-rbac-proxy-web/0.log" Jun 15 09:34:30.716165 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:30.716145 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_38d75778-28a1-4c29-bcb2-46470d15cae4/kube-rbac-proxy/0.log" Jun 15 09:34:30.738438 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:30.738422 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_38d75778-28a1-4c29-bcb2-46470d15cae4/kube-rbac-proxy-thanos/0.log" Jun 15 09:34:30.757899 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:30.757855 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_38d75778-28a1-4c29-bcb2-46470d15cae4/init-config-reloader/0.log" Jun 15 09:34:33.068503 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:33.068462 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p"] Jun 15 09:34:33.071974 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:33.071955 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p" Jun 15 09:34:33.074639 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:33.074612 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dqdjg\"/\"openshift-service-ca.crt\"" Jun 15 09:34:33.075762 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:33.075739 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dqdjg\"/\"kube-root-ca.crt\"" Jun 15 09:34:33.075880 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:33.075849 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dqdjg\"/\"default-dockercfg-n8dgm\"" Jun 15 09:34:33.084595 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:33.084576 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p"] Jun 15 09:34:33.110643 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:33.110615 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/822592d2-26a9-4496-82f6-e56b264201f0-sys\") pod \"perf-node-gather-daemonset-w9v8p\" (UID: \"822592d2-26a9-4496-82f6-e56b264201f0\") " pod="openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p" Jun 15 09:34:33.110815 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:33.110657 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/822592d2-26a9-4496-82f6-e56b264201f0-proc\") pod \"perf-node-gather-daemonset-w9v8p\" (UID: \"822592d2-26a9-4496-82f6-e56b264201f0\") " pod="openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p" Jun 15 09:34:33.110815 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:33.110676 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/822592d2-26a9-4496-82f6-e56b264201f0-lib-modules\") pod \"perf-node-gather-daemonset-w9v8p\" (UID: \"822592d2-26a9-4496-82f6-e56b264201f0\") " pod="openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p" Jun 15 09:34:33.110815 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:33.110716 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/822592d2-26a9-4496-82f6-e56b264201f0-podres\") pod \"perf-node-gather-daemonset-w9v8p\" (UID: \"822592d2-26a9-4496-82f6-e56b264201f0\") " pod="openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p" Jun 15 09:34:33.110815 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:33.110750 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbnzp\" (UniqueName: \"kubernetes.io/projected/822592d2-26a9-4496-82f6-e56b264201f0-kube-api-access-pbnzp\") pod \"perf-node-gather-daemonset-w9v8p\" (UID: \"822592d2-26a9-4496-82f6-e56b264201f0\") " pod="openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p" Jun 15 09:34:33.211388 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:33.211364 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/822592d2-26a9-4496-82f6-e56b264201f0-sys\") pod \"perf-node-gather-daemonset-w9v8p\" (UID: \"822592d2-26a9-4496-82f6-e56b264201f0\") " pod="openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p" Jun 15 09:34:33.211544 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:33.211410 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/822592d2-26a9-4496-82f6-e56b264201f0-proc\") pod \"perf-node-gather-daemonset-w9v8p\" (UID: \"822592d2-26a9-4496-82f6-e56b264201f0\") " pod="openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p" Jun 15 09:34:33.211544 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:33.211452 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/822592d2-26a9-4496-82f6-e56b264201f0-lib-modules\") pod \"perf-node-gather-daemonset-w9v8p\" (UID: \"822592d2-26a9-4496-82f6-e56b264201f0\") " pod="openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p" Jun 15 09:34:33.211544 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:33.211478 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/822592d2-26a9-4496-82f6-e56b264201f0-proc\") pod \"perf-node-gather-daemonset-w9v8p\" (UID: \"822592d2-26a9-4496-82f6-e56b264201f0\") " pod="openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p" Jun 15 09:34:33.211544 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:33.211495 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/822592d2-26a9-4496-82f6-e56b264201f0-podres\") pod \"perf-node-gather-daemonset-w9v8p\" (UID: \"822592d2-26a9-4496-82f6-e56b264201f0\") " pod="openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p" Jun 15 09:34:33.211544 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:33.211532 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbnzp\" (UniqueName: \"kubernetes.io/projected/822592d2-26a9-4496-82f6-e56b264201f0-kube-api-access-pbnzp\") pod \"perf-node-gather-daemonset-w9v8p\" (UID: \"822592d2-26a9-4496-82f6-e56b264201f0\") " pod="openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p" Jun 15 09:34:33.211763 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:33.211533 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/822592d2-26a9-4496-82f6-e56b264201f0-sys\") pod \"perf-node-gather-daemonset-w9v8p\" (UID: \"822592d2-26a9-4496-82f6-e56b264201f0\") " pod="openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p" Jun 15 09:34:33.211763 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:33.211638 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/822592d2-26a9-4496-82f6-e56b264201f0-lib-modules\") pod \"perf-node-gather-daemonset-w9v8p\" (UID: \"822592d2-26a9-4496-82f6-e56b264201f0\") " pod="openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p" Jun 15 09:34:33.211763 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:33.211653 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/822592d2-26a9-4496-82f6-e56b264201f0-podres\") pod \"perf-node-gather-daemonset-w9v8p\" (UID: \"822592d2-26a9-4496-82f6-e56b264201f0\") " pod="openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p" Jun 15 09:34:33.220629 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:33.220607 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbnzp\" (UniqueName: \"kubernetes.io/projected/822592d2-26a9-4496-82f6-e56b264201f0-kube-api-access-pbnzp\") pod \"perf-node-gather-daemonset-w9v8p\" (UID: \"822592d2-26a9-4496-82f6-e56b264201f0\") " pod="openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p" Jun 15 09:34:33.381875 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:33.381779 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p" Jun 15 09:34:33.510248 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:33.510223 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p"] Jun 15 09:34:33.512456 ip-10-0-129-143 kubenswrapper[2562]: W0615 09:34:33.512431 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod822592d2_26a9_4496_82f6_e56b264201f0.slice/crio-51072fc0e23448ea8cd39ff3b431bee7e00a59a07c0b5951bbcdaaf34a481fbf WatchSource:0}: Error finding container 51072fc0e23448ea8cd39ff3b431bee7e00a59a07c0b5951bbcdaaf34a481fbf: Status 404 returned error can't find the container with id 51072fc0e23448ea8cd39ff3b431bee7e00a59a07c0b5951bbcdaaf34a481fbf Jun 15 09:34:34.015988 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:34.015897 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p" event={"ID":"822592d2-26a9-4496-82f6-e56b264201f0","Type":"ContainerStarted","Data":"b7f3ab2c4d4d8cb7c1a835a50d81781662c44d79a54a9d40400c8641ab964e71"} Jun 15 09:34:34.015988 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:34.015952 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p" event={"ID":"822592d2-26a9-4496-82f6-e56b264201f0","Type":"ContainerStarted","Data":"51072fc0e23448ea8cd39ff3b431bee7e00a59a07c0b5951bbcdaaf34a481fbf"} Jun 15 09:34:34.016221 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:34.016026 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p" Jun 15 09:34:34.031796 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:34.031745 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p" podStartSLOduration=1.031731453 podStartE2EDuration="1.031731453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 09:34:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 09:34:34.030229956 +0000 UTC m=+958.593652262" watchObservedRunningTime="2026-06-15 09:34:34.031731453 +0000 UTC m=+958.595153758" Jun 15 09:34:35.376385 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:35.376362 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rr6xd_695a033a-993a-4bd2-9f73-3f979df52254/dns/0.log" Jun 15 09:34:35.413516 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:35.413493 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rr6xd_695a033a-993a-4bd2-9f73-3f979df52254/kube-rbac-proxy/0.log" Jun 15 09:34:35.482530 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:35.482497 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jlhbm_eac64cde-28f4-49be-8543-42e3b4ed898f/dns-node-resolver/0.log" Jun 15 09:34:36.047881 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:36.047855 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lfzxm_a11b7428-db5b-4672-97f0-4cfaf32de873/node-ca/0.log" Jun 15 09:34:37.066442 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:37.066366 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bcbb8b488-cr726_28a85a02-7443-41b3-a429-ef011dc7f3c2/kube-auth-proxy/0.log" Jun 15 09:34:37.091080 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:37.091055 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-qxvtx_30d60e63-17db-47e9-800c-2c5bd368678e/istio-proxy/1.log" Jun 15 09:34:37.091459 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:37.091447 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-qxvtx_30d60e63-17db-47e9-800c-2c5bd368678e/istio-proxy/0.log" Jun 15 09:34:37.110947 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:37.110928 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-hl8fp_c80e6536-a697-4f6e-b3ab-b4f39b5d385d/payload-pre-processing/0.log" Jun 15 09:34:37.132857 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:37.132824 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-fqnxb_4bb7231f-d2c5-421c-9f83-4254c4799157/payload-processing/0.log" Jun 15 09:34:37.785981 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:37.785950 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gwnsl_22891e8c-12f1-48c2-910f-5015573cdad8/serve-healthcheck-canary/0.log" Jun 15 09:34:38.400037 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:38.400010 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kz4mb_79f2f4a8-307d-48e2-9c31-83bcd1636183/kube-rbac-proxy/0.log" Jun 15 09:34:38.418294 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:38.418270 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kz4mb_79f2f4a8-307d-48e2-9c31-83bcd1636183/exporter/0.log" Jun 15 09:34:38.437953 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:38.437927 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kz4mb_79f2f4a8-307d-48e2-9c31-83bcd1636183/extractor/0.log" Jun 15 09:34:40.030219 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:40.030188 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dqdjg/perf-node-gather-daemonset-w9v8p" Jun 15 09:34:40.800111 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:40.800085 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-697554dff8-66j4v_93a0f85d-5bf7-4028-b63e-dfa26233f10c/maas-api/0.log" Jun 15 09:34:40.848271 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:40.848246 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-767f9856ff-sxbgt_85021cfa-1559-4ece-81f3-556fe5004e5a/manager/0.log" Jun 15 09:34:40.994270 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:40.994244 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5659c68bbf-jjfvb_aafe0dfe-75d7-42b6-979c-a466e4edf40b/manager/0.log" Jun 15 09:34:41.012421 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:41.012393 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5659c68bbf-jjfvb_aafe0dfe-75d7-42b6-979c-a466e4edf40b/copy-manifests/0.log" Jun 15 09:34:41.076980 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:41.076957 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-6p2lm_6f42c62e-7931-4fb9-857f-b334967d9e9a/postgres/0.log" Jun 15 09:34:42.817650 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:42.817627 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-77b99946d4-8864g_28295990-b72d-473e-b481-1330cffbeaa7/manager/0.log" Jun 15 09:34:47.529532 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:47.529501 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bcbb8b488-cr726_28a85a02-7443-41b3-a429-ef011dc7f3c2/kube-auth-proxy/0.log" Jun 15 09:34:47.550946 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:47.550921 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-qxvtx_30d60e63-17db-47e9-800c-2c5bd368678e/istio-proxy/1.log" Jun 15 09:34:47.551303 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:47.551288 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-qxvtx_30d60e63-17db-47e9-800c-2c5bd368678e/istio-proxy/0.log" Jun 15 09:34:47.560717 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:47.560695 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-hl8fp_c80e6536-a697-4f6e-b3ab-b4f39b5d385d/payload-pre-processing/0.log" Jun 15 09:34:47.575589 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:47.575566 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-fqnxb_4bb7231f-d2c5-421c-9f83-4254c4799157/payload-processing/0.log" Jun 15 09:34:47.900366 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:47.900332 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d4bcd4867-kdt7s_4bb39dda-ff30-41eb-a023-8866e8df5889/migrator/0.log" Jun 15 09:34:47.921391 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:47.921357 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d4bcd4867-kdt7s_4bb39dda-ff30-41eb-a023-8866e8df5889/graceful-termination/0.log" Jun 15 09:34:48.745544 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:48.745517 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bcbb8b488-cr726_28a85a02-7443-41b3-a429-ef011dc7f3c2/kube-auth-proxy/0.log" Jun 15 09:34:48.770609 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:48.770583 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-qxvtx_30d60e63-17db-47e9-800c-2c5bd368678e/istio-proxy/1.log" Jun 15 09:34:48.770866 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:48.770830 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-qxvtx_30d60e63-17db-47e9-800c-2c5bd368678e/istio-proxy/0.log" Jun 15 09:34:48.782633 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:48.782614 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-hl8fp_c80e6536-a697-4f6e-b3ab-b4f39b5d385d/payload-pre-processing/0.log" Jun 15 09:34:48.800196 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:48.800176 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-fqnxb_4bb7231f-d2c5-421c-9f83-4254c4799157/payload-processing/0.log" Jun 15 09:34:49.449634 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:49.449561 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-44gsl_92f911d1-972a-4024-9cd7-311529a59fb7/kube-multus-additional-cni-plugins/0.log" Jun 15 09:34:49.469226 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:49.469204 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-44gsl_92f911d1-972a-4024-9cd7-311529a59fb7/egress-router-binary-copy/0.log" Jun 15 09:34:49.487820 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:49.487797 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-44gsl_92f911d1-972a-4024-9cd7-311529a59fb7/cni-plugins/0.log" Jun 15 09:34:49.505867 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:49.505828 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-44gsl_92f911d1-972a-4024-9cd7-311529a59fb7/bond-cni-plugin/0.log" Jun 15 09:34:49.525118 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:49.525097 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-44gsl_92f911d1-972a-4024-9cd7-311529a59fb7/routeoverride-cni/0.log" Jun 15 09:34:49.543984 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:49.543962 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-44gsl_92f911d1-972a-4024-9cd7-311529a59fb7/whereabouts-cni-bincopy/0.log" Jun 15 09:34:49.563730 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:49.563711 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-44gsl_92f911d1-972a-4024-9cd7-311529a59fb7/whereabouts-cni/0.log" Jun 15 09:34:49.944091 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:49.944063 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-sxdtv_2699a480-5720-4d12-ab42-7c30a489e7d7/kube-multus/0.log" Jun 15 09:34:50.012966 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:50.012938 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7trxt_67c71a8c-71f7-4494-a8cd-fd7b2af8ab51/network-metrics-daemon/0.log" Jun 15 09:34:50.030900 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:50.030878 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7trxt_67c71a8c-71f7-4494-a8cd-fd7b2af8ab51/kube-rbac-proxy/0.log" Jun 15 09:34:50.400593 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:50.400555 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bcbb8b488-cr726_28a85a02-7443-41b3-a429-ef011dc7f3c2/kube-auth-proxy/0.log" Jun 15 09:34:50.417588 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:50.417558 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-qxvtx_30d60e63-17db-47e9-800c-2c5bd368678e/istio-proxy/1.log" Jun 15 09:34:50.418040 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:50.418022 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-qxvtx_30d60e63-17db-47e9-800c-2c5bd368678e/istio-proxy/0.log" Jun 15 09:34:50.426267 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:50.426244 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-hl8fp_c80e6536-a697-4f6e-b3ab-b4f39b5d385d/payload-pre-processing/0.log" Jun 15 09:34:50.439973 ip-10-0-129-143 kubenswrapper[2562]: I0615 09:34:50.439950 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-fqnxb_4bb7231f-d2c5-421c-9f83-4254c4799157/payload-processing/0.log"