Jun 08 18:12:46.028279 ip-10-0-138-194 systemd[1]: Starting Kubernetes Kubelet... Jun 08 18:12:46.437056 ip-10-0-138-194 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 18:12:46.437056 ip-10-0-138-194 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 08 18:12:46.437056 ip-10-0-138-194 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 18:12:46.437056 ip-10-0-138-194 kubenswrapper[2572]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 08 18:12:46.437056 ip-10-0-138-194 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 18:12:46.439663 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.439567 2572 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 08 18:12:46.445678 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445647 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 18:12:46.445678 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445670 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 18:12:46.445678 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445674 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 18:12:46.445678 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445677 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 18:12:46.445678 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445680 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 18:12:46.445678 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445683 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 18:12:46.445678 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445686 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 18:12:46.445678 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445689 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 18:12:46.445981 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445692 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 18:12:46.445981 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445695 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 18:12:46.445981 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445698 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 18:12:46.445981 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445700 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 18:12:46.445981 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445703 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 18:12:46.445981 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445706 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 18:12:46.445981 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445708 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 18:12:46.445981 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445711 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 18:12:46.445981 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445714 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 18:12:46.445981 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445717 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 18:12:46.445981 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445719 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 18:12:46.445981 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445722 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 18:12:46.445981 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445724 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 18:12:46.445981 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445727 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 18:12:46.445981 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445730 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 18:12:46.445981 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445732 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 18:12:46.445981 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445735 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 18:12:46.445981 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445737 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 18:12:46.445981 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445740 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 18:12:46.446523 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445743 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 18:12:46.446523 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445745 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 18:12:46.446523 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445748 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 18:12:46.446523 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445750 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 18:12:46.446523 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445753 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 18:12:46.446523 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445762 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 18:12:46.446523 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445767 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 18:12:46.446523 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445771 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 18:12:46.446523 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445774 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 18:12:46.446523 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445777 2572 feature_gate.go:328] unrecognized feature gate: Example Jun 08 18:12:46.446523 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445780 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 18:12:46.446523 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445783 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 18:12:46.446523 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445785 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 18:12:46.446523 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445788 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 18:12:46.446523 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445792 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 18:12:46.446523 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445794 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 18:12:46.446523 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445797 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 18:12:46.446523 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445800 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 18:12:46.446523 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445803 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 18:12:46.446523 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445805 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 18:12:46.447038 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445808 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 18:12:46.447038 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445811 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 18:12:46.447038 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445813 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 18:12:46.447038 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445816 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 18:12:46.447038 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445819 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 18:12:46.447038 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445821 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 18:12:46.447038 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445824 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 18:12:46.447038 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445827 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 18:12:46.447038 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445830 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 18:12:46.447038 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445832 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 18:12:46.447038 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445835 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 18:12:46.447038 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445837 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 18:12:46.447038 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445840 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 18:12:46.447038 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445842 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 18:12:46.447038 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445845 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 18:12:46.447038 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445848 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 18:12:46.447038 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445850 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 18:12:46.447038 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445853 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 18:12:46.447038 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445856 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 18:12:46.447038 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445859 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 18:12:46.447562 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445863 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 18:12:46.447562 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445867 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 18:12:46.447562 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445870 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 18:12:46.447562 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445873 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 18:12:46.447562 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445875 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 18:12:46.447562 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445878 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 18:12:46.447562 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445882 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 18:12:46.447562 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445885 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 18:12:46.447562 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445887 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 18:12:46.447562 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445890 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 18:12:46.447562 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445892 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 18:12:46.447562 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445895 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 18:12:46.447562 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445897 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 18:12:46.447562 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445900 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 18:12:46.447562 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445913 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 18:12:46.447562 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445916 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 18:12:46.447562 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445925 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 18:12:46.447562 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445927 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 18:12:46.447562 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445930 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 18:12:46.447562 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445933 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 18:12:46.448135 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445937 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 18:12:46.448135 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445940 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 18:12:46.448135 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445943 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 18:12:46.448135 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445945 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 18:12:46.448135 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445948 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 18:12:46.448135 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445951 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 18:12:46.448135 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445953 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 18:12:46.448135 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445971 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 18:12:46.448135 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445974 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 18:12:46.448135 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445977 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 18:12:46.448135 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445981 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 18:12:46.448135 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.445985 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 18:12:46.448135 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446736 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 18:12:46.448135 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446744 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 18:12:46.448135 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446747 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 18:12:46.448135 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446750 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 18:12:46.448135 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446752 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 18:12:46.448135 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446757 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 18:12:46.448135 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446761 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 18:12:46.448601 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446764 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 18:12:46.448601 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446767 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 18:12:46.448601 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446770 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 18:12:46.448601 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446772 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 18:12:46.448601 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446775 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 18:12:46.448601 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446778 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 18:12:46.448601 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446780 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 18:12:46.448601 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446783 2572 feature_gate.go:328] unrecognized feature gate: Example Jun 08 18:12:46.448601 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446786 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 18:12:46.448601 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446788 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 18:12:46.448601 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446794 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 18:12:46.448601 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446797 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 18:12:46.448601 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446800 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 18:12:46.448601 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446803 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 18:12:46.448601 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446805 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 18:12:46.448601 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446808 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 18:12:46.448601 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446811 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 18:12:46.448601 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446813 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 18:12:46.448601 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446816 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 18:12:46.448601 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446819 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 18:12:46.449126 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446821 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 18:12:46.449126 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446824 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 18:12:46.449126 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446827 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 18:12:46.449126 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446830 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 18:12:46.449126 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446833 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 18:12:46.449126 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446835 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 18:12:46.449126 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446838 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 18:12:46.449126 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446840 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 18:12:46.449126 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446843 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 18:12:46.449126 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446846 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 18:12:46.449126 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446851 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 18:12:46.449126 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446854 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 18:12:46.449126 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446857 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 18:12:46.449126 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446860 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 18:12:46.449126 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446864 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 18:12:46.449126 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446866 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 18:12:46.449126 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446870 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 18:12:46.449126 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446874 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 18:12:46.449126 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446877 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 18:12:46.449599 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446879 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 18:12:46.449599 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446882 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 18:12:46.449599 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446889 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 18:12:46.449599 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446891 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 18:12:46.449599 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446895 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 18:12:46.449599 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446898 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 18:12:46.449599 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446901 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 18:12:46.449599 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446904 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 18:12:46.449599 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446907 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 18:12:46.449599 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446909 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 18:12:46.449599 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446912 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 18:12:46.449599 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446915 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 18:12:46.449599 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446918 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 18:12:46.449599 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446920 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 18:12:46.449599 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446923 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 18:12:46.449599 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446926 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 18:12:46.449599 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446928 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 18:12:46.449599 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446931 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 18:12:46.449599 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446934 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 18:12:46.449599 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446936 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 18:12:46.450104 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446939 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 18:12:46.450104 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446941 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 18:12:46.450104 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446944 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 18:12:46.450104 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446946 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 18:12:46.450104 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446951 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 18:12:46.450104 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446953 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 18:12:46.450104 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446974 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 18:12:46.450104 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446978 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 18:12:46.450104 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446981 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 18:12:46.450104 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446983 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 18:12:46.450104 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446986 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 18:12:46.450104 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446989 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 18:12:46.450104 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446991 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 18:12:46.450104 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446994 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 18:12:46.450104 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.446998 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 18:12:46.450104 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447000 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 18:12:46.450104 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447003 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 18:12:46.450104 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447006 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 18:12:46.450104 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447009 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 18:12:46.450104 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447013 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 18:12:46.450593 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447015 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 18:12:46.450593 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447018 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 18:12:46.450593 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447020 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 18:12:46.450593 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447023 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 18:12:46.450593 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447025 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 18:12:46.450593 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447028 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 18:12:46.450593 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447030 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 18:12:46.450593 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447033 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 18:12:46.450593 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447036 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 18:12:46.450593 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447039 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 18:12:46.450593 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447041 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 18:12:46.450593 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447044 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 18:12:46.450593 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447047 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 18:12:46.450593 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447452 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 18:12:46.450593 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447457 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 18:12:46.450593 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447461 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 18:12:46.450593 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447463 2572 feature_gate.go:328] unrecognized feature gate: Example Jun 08 18:12:46.450593 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447466 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 18:12:46.450593 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447469 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 18:12:46.450593 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447471 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 18:12:46.451111 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447474 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 18:12:46.451111 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447477 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 18:12:46.451111 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447479 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 18:12:46.451111 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447482 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 18:12:46.451111 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447484 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 18:12:46.451111 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447487 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 18:12:46.451111 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447492 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 18:12:46.451111 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447495 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 18:12:46.451111 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447498 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 18:12:46.451111 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447502 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 18:12:46.451111 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447505 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 18:12:46.451111 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447509 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 18:12:46.451111 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447513 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 18:12:46.451111 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447517 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 18:12:46.451111 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447519 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 18:12:46.451111 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447522 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 18:12:46.451111 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447525 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 18:12:46.451111 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447527 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 18:12:46.451552 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447532 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 18:12:46.451552 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447535 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 18:12:46.451552 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447538 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 18:12:46.451552 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447541 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 18:12:46.451552 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447544 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 18:12:46.451552 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447547 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 18:12:46.451552 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447549 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 18:12:46.451552 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447552 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 18:12:46.451552 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447555 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 18:12:46.451552 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447558 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 18:12:46.451552 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447561 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 18:12:46.451552 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447564 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 18:12:46.451552 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447567 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 18:12:46.451552 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447570 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 18:12:46.451552 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447573 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 18:12:46.451552 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447575 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 18:12:46.451552 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447578 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 18:12:46.451552 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447581 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 18:12:46.451552 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447584 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 18:12:46.452040 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447586 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 18:12:46.452040 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447589 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 18:12:46.452040 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447591 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 18:12:46.452040 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447594 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 18:12:46.452040 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447596 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 18:12:46.452040 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447599 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 18:12:46.452040 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447606 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 18:12:46.452040 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447609 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 18:12:46.452040 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447611 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 18:12:46.452040 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447614 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 18:12:46.452040 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447617 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 18:12:46.452040 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447620 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 18:12:46.452040 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447622 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 18:12:46.452040 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447625 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 18:12:46.452040 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447627 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 18:12:46.452040 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447629 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 18:12:46.452040 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447632 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 18:12:46.452040 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447635 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 18:12:46.452040 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447637 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 18:12:46.452040 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447639 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 18:12:46.452569 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447642 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 18:12:46.452569 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447645 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 18:12:46.452569 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447647 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 18:12:46.452569 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447650 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 18:12:46.452569 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447652 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 18:12:46.452569 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447655 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 18:12:46.452569 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447658 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 18:12:46.452569 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447660 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 18:12:46.452569 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447663 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 18:12:46.452569 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447665 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 18:12:46.452569 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447668 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 18:12:46.452569 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447670 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 18:12:46.452569 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447675 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 18:12:46.452569 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447677 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 18:12:46.452569 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447680 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 18:12:46.452569 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447682 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 18:12:46.452569 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447685 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 18:12:46.452569 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447687 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 18:12:46.452569 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447692 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 18:12:46.452569 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447694 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 18:12:46.453080 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447697 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 18:12:46.453080 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447700 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 18:12:46.453080 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447702 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 18:12:46.453080 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447704 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 18:12:46.453080 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447707 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 18:12:46.453080 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447709 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 18:12:46.453080 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447712 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 18:12:46.453080 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447715 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 18:12:46.453080 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447717 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 18:12:46.453080 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447720 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 18:12:46.453080 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447722 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 18:12:46.453080 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447725 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 18:12:46.453080 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447727 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 18:12:46.453080 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447730 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 18:12:46.453080 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.447732 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 18:12:46.453080 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447815 2572 flags.go:64] FLAG: --address="0.0.0.0" Jun 08 18:12:46.453080 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447826 2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 08 18:12:46.453080 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447836 2572 flags.go:64] FLAG: --anonymous-auth="true" Jun 08 18:12:46.453080 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447842 2572 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 08 18:12:46.453080 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447849 2572 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 08 18:12:46.453080 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447854 2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447859 2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447864 2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447867 2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447870 2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447874 2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447877 2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447880 2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447883 2572 flags.go:64] FLAG: --cgroup-root="" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447886 2572 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447891 2572 flags.go:64] FLAG: --client-ca-file="" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447895 2572 flags.go:64] FLAG: --cloud-provider="external" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447898 2572 flags.go:64] FLAG: --cluster-dns="[]" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447903 2572 flags.go:64] FLAG: --cluster-domain="" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447905 2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447908 2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447911 2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447915 2572 flags.go:64] FLAG: --container-log-max-files="5" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447919 2572 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447922 2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447925 2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447929 2572 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447931 2572 flags.go:64] FLAG: --contention-profiling="false" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447935 2572 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 08 18:12:46.453591 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447937 2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447941 2572 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447944 2572 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447948 2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447952 2572 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447968 2572 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447972 2572 flags.go:64] FLAG: --enable-load-reader="false" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447975 2572 flags.go:64] FLAG: --enable-server="true" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447978 2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447983 2572 flags.go:64] FLAG: --event-burst="100" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447986 2572 flags.go:64] FLAG: --event-qps="50" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447989 2572 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447992 2572 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447994 2572 flags.go:64] FLAG: --eviction-hard="" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.447999 2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448002 2572 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448005 2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448008 2572 flags.go:64] FLAG: --eviction-soft="" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448012 2572 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448015 2572 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448019 2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448022 2572 flags.go:64] FLAG: --experimental-mounter-path="" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448025 2572 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448028 2572 flags.go:64] FLAG: --fail-swap-on="true" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448030 2572 flags.go:64] FLAG: --feature-gates="" Jun 08 18:12:46.454185 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448035 2572 flags.go:64] FLAG: --file-check-frequency="20s" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448037 2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448040 2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448043 2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448047 2572 flags.go:64] FLAG: --healthz-port="10248" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448050 2572 flags.go:64] FLAG: --help="false" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448053 2572 flags.go:64] FLAG: --hostname-override="ip-10-0-138-194.ec2.internal" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448056 2572 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448059 2572 flags.go:64] FLAG: --http-check-frequency="20s" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448062 2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448065 2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448071 2572 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448078 2572 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448081 2572 flags.go:64] FLAG: --image-service-endpoint="" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448084 2572 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448087 2572 flags.go:64] FLAG: --kube-api-burst="100" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448090 2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448093 2572 flags.go:64] FLAG: --kube-api-qps="50" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448096 2572 flags.go:64] FLAG: --kube-reserved="" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448099 2572 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448102 2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448105 2572 flags.go:64] FLAG: --kubelet-cgroups="" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448108 2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448111 2572 flags.go:64] FLAG: --lock-file="" Jun 08 18:12:46.454882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448114 2572 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448118 2572 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448121 2572 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448127 2572 flags.go:64] FLAG: --log-json-split-stream="false" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448131 2572 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448134 2572 flags.go:64] FLAG: --log-text-split-stream="false" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448137 2572 flags.go:64] FLAG: --logging-format="text" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448141 2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448144 2572 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448147 2572 flags.go:64] FLAG: --manifest-url="" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448150 2572 flags.go:64] FLAG: --manifest-url-header="" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448154 2572 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448158 2572 flags.go:64] FLAG: --max-open-files="1000000" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448162 2572 flags.go:64] FLAG: --max-pods="110" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448165 2572 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448168 2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448171 2572 flags.go:64] FLAG: --memory-manager-policy="None" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448174 2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448177 2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448181 2572 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448184 2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448192 2572 flags.go:64] FLAG: --node-status-max-images="50" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448195 2572 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448198 2572 flags.go:64] FLAG: --oom-score-adj="-999" Jun 08 18:12:46.455502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448201 2572 flags.go:64] FLAG: --pod-cidr="" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448204 2572 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448209 2572 flags.go:64] FLAG: --pod-manifest-path="" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448212 2572 flags.go:64] FLAG: --pod-max-pids="-1" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448215 2572 flags.go:64] FLAG: --pods-per-core="0" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448218 2572 flags.go:64] FLAG: --port="10250" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448221 2572 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448224 2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0dd416980105e510e" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448227 2572 flags.go:64] FLAG: --qos-reserved="" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448231 2572 flags.go:64] FLAG: --read-only-port="10255" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448234 2572 flags.go:64] FLAG: --register-node="true" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448237 2572 flags.go:64] FLAG: --register-with-taints="" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448241 2572 flags.go:64] FLAG: --registry-burst="10" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448244 2572 flags.go:64] FLAG: --registry-qps="5" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448247 2572 flags.go:64] FLAG: --reserved-cpus="" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448250 2572 flags.go:64] FLAG: --reserved-memory="" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448254 2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448257 2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448260 2572 flags.go:64] FLAG: --rotate-certificates="false" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448263 2572 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448266 2572 flags.go:64] FLAG: --runonce="false" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448269 2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448272 2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448275 2572 flags.go:64] FLAG: --seccomp-default="false" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448278 2572 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 08 18:12:46.456114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448280 2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448283 2572 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448286 2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448290 2572 flags.go:64] FLAG: --storage-driver-password="root" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448293 2572 flags.go:64] FLAG: --storage-driver-secure="false" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448297 2572 flags.go:64] FLAG: --storage-driver-table="stats" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448300 2572 flags.go:64] FLAG: --storage-driver-user="root" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448303 2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448306 2572 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448308 2572 flags.go:64] FLAG: --system-cgroups="" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448311 2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448316 2572 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448319 2572 flags.go:64] FLAG: --tls-cert-file="" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448322 2572 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448327 2572 flags.go:64] FLAG: --tls-min-version="" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448330 2572 flags.go:64] FLAG: --tls-private-key-file="" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448333 2572 flags.go:64] FLAG: --topology-manager-policy="none" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448336 2572 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448339 2572 flags.go:64] FLAG: --topology-manager-scope="container" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448342 2572 flags.go:64] FLAG: --v="2" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448346 2572 flags.go:64] FLAG: --version="false" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448351 2572 flags.go:64] FLAG: --vmodule="" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448355 2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.448358 2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448481 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 18:12:46.456727 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448485 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 18:12:46.457362 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448488 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 18:12:46.457362 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448492 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 18:12:46.457362 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448496 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 18:12:46.457362 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448499 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 18:12:46.457362 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448502 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 18:12:46.457362 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448504 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 18:12:46.457362 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448507 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 18:12:46.457362 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448511 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 18:12:46.457362 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448515 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 18:12:46.457362 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448519 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 18:12:46.457362 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448522 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 18:12:46.457362 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448525 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 18:12:46.457362 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448528 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 18:12:46.457362 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448531 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 18:12:46.457362 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448533 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 18:12:46.457362 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448536 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 18:12:46.457362 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448539 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 18:12:46.457362 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448542 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 18:12:46.457362 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448544 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 18:12:46.457834 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448547 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 18:12:46.457834 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448549 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 18:12:46.457834 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448552 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 18:12:46.457834 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448555 2572 feature_gate.go:328] unrecognized feature gate: Example Jun 08 18:12:46.457834 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448558 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 18:12:46.457834 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448560 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 18:12:46.457834 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448563 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 18:12:46.457834 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448566 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 18:12:46.457834 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448568 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 18:12:46.457834 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448571 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 18:12:46.457834 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448574 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 18:12:46.457834 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448577 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 18:12:46.457834 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448580 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 18:12:46.457834 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448582 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 18:12:46.457834 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448585 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 18:12:46.457834 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448587 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 18:12:46.457834 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448590 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 18:12:46.457834 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448593 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 18:12:46.457834 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448596 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 18:12:46.457834 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448598 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 18:12:46.458343 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448601 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 18:12:46.458343 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448604 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 18:12:46.458343 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448607 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 18:12:46.458343 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448612 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 18:12:46.458343 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448615 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 18:12:46.458343 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448619 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 18:12:46.458343 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448622 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 18:12:46.458343 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448625 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 18:12:46.458343 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448628 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 18:12:46.458343 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448631 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 18:12:46.458343 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448633 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 18:12:46.458343 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448636 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 18:12:46.458343 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448638 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 18:12:46.458343 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448641 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 18:12:46.458343 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448644 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 18:12:46.458343 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448647 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 18:12:46.458343 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448649 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 18:12:46.458343 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448652 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 18:12:46.458343 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448655 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 18:12:46.458343 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448657 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 18:12:46.458844 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448659 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 18:12:46.458844 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448662 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 18:12:46.458844 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448672 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 18:12:46.458844 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448675 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 18:12:46.458844 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448678 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 18:12:46.458844 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448680 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 18:12:46.458844 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448682 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 18:12:46.458844 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448685 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 18:12:46.458844 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448688 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 18:12:46.458844 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448690 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 18:12:46.458844 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448693 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 18:12:46.458844 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448695 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 18:12:46.458844 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448698 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 18:12:46.458844 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448700 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 18:12:46.458844 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448704 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 18:12:46.458844 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448707 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 18:12:46.458844 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448710 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 18:12:46.458844 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448712 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 18:12:46.458844 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448715 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 18:12:46.458844 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448718 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 18:12:46.459430 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448720 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 18:12:46.459430 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448723 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 18:12:46.459430 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448726 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 18:12:46.459430 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448728 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 18:12:46.459430 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448731 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 18:12:46.459430 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448733 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 18:12:46.459430 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448736 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 18:12:46.459430 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448739 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 18:12:46.459430 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448742 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 18:12:46.459430 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448745 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 18:12:46.459430 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448747 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 18:12:46.459430 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448750 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 18:12:46.459430 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448752 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 18:12:46.459430 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448755 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 18:12:46.459430 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448757 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 18:12:46.459430 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448760 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 18:12:46.459430 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448763 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 18:12:46.459430 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.448766 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 18:12:46.459884 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.449465 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 18:12:46.459884 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.456409 2572 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 08 18:12:46.459884 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.456433 2572 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 08 18:12:46.459884 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.457208 2572 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 08 18:12:46.459884 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.457217 2572 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 08 18:12:46.459884 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457282 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 18:12:46.459884 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457288 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 18:12:46.459884 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457291 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 18:12:46.459884 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457294 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 18:12:46.459884 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457297 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 18:12:46.459884 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457300 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 18:12:46.459884 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457304 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 18:12:46.459884 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457308 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 18:12:46.459884 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457311 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 18:12:46.460257 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457314 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 18:12:46.460257 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457317 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 18:12:46.460257 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457320 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 18:12:46.460257 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457323 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 18:12:46.460257 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457326 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 18:12:46.460257 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457329 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 18:12:46.460257 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457332 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 18:12:46.460257 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457335 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 18:12:46.460257 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457337 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 18:12:46.460257 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457340 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 18:12:46.460257 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457343 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 18:12:46.460257 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457346 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 18:12:46.460257 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457348 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 18:12:46.460257 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457351 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 18:12:46.460257 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457353 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 18:12:46.460257 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457356 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 18:12:46.460257 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457358 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 18:12:46.460257 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457361 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 18:12:46.460257 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457363 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 18:12:46.460257 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457367 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 18:12:46.460745 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457378 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 18:12:46.460745 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457381 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 18:12:46.460745 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457384 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 18:12:46.460745 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457387 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 18:12:46.460745 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457390 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 18:12:46.460745 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457392 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 18:12:46.460745 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457395 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 18:12:46.460745 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457397 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 18:12:46.460745 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457400 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 18:12:46.460745 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457403 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 18:12:46.460745 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457405 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 18:12:46.460745 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457408 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 18:12:46.460745 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457410 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 18:12:46.460745 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457413 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 18:12:46.460745 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457416 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 18:12:46.460745 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457419 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 18:12:46.460745 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457422 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 18:12:46.460745 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457424 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 18:12:46.460745 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457427 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 18:12:46.460745 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457429 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 18:12:46.461253 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457432 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 18:12:46.461253 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457434 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 18:12:46.461253 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457437 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 18:12:46.461253 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457439 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 18:12:46.461253 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457442 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 18:12:46.461253 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457444 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 18:12:46.461253 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457448 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 18:12:46.461253 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457451 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 18:12:46.461253 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457453 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 18:12:46.461253 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457456 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 18:12:46.461253 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457459 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 18:12:46.461253 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457461 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 18:12:46.461253 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457465 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 18:12:46.461253 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457469 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 18:12:46.461253 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457471 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 18:12:46.461253 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457474 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 18:12:46.461253 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457476 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 18:12:46.461253 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457479 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 18:12:46.461253 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457481 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 18:12:46.461253 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457484 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 18:12:46.461779 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457486 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 18:12:46.461779 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457489 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 18:12:46.461779 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457491 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 18:12:46.461779 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457494 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 18:12:46.461779 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457496 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 18:12:46.461779 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457499 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 18:12:46.461779 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457501 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 18:12:46.461779 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457504 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 18:12:46.461779 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457506 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 18:12:46.461779 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457509 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 18:12:46.461779 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457512 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 18:12:46.461779 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457516 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 18:12:46.461779 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457519 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 18:12:46.461779 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457523 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 18:12:46.461779 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457526 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 18:12:46.461779 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457529 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 18:12:46.461779 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457531 2572 feature_gate.go:328] unrecognized feature gate: Example Jun 08 18:12:46.461779 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457535 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 18:12:46.461779 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457539 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 18:12:46.462280 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457542 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 18:12:46.462280 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457545 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 18:12:46.462280 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457548 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 18:12:46.462280 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457551 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 18:12:46.462280 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457554 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 18:12:46.462280 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457557 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 18:12:46.462280 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457559 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 18:12:46.462280 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457563 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 18:12:46.462280 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457565 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 18:12:46.462280 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457567 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 18:12:46.462280 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457570 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 18:12:46.462280 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.457575 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 18:12:46.462280 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457707 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 18:12:46.462280 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457711 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 18:12:46.462280 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457714 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 18:12:46.462656 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457717 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 18:12:46.462656 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457719 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 18:12:46.462656 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457722 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 18:12:46.462656 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457725 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 18:12:46.462656 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457728 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 18:12:46.462656 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457731 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 18:12:46.462656 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457733 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 18:12:46.462656 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457736 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 18:12:46.462656 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457739 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 18:12:46.462656 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457741 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 18:12:46.462656 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457745 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 18:12:46.462656 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457747 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 18:12:46.462656 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457750 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 18:12:46.462656 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457752 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 18:12:46.462656 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457755 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 18:12:46.462656 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457758 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 18:12:46.462656 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457760 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 18:12:46.462656 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457763 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 18:12:46.462656 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457766 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 18:12:46.462656 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457769 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 18:12:46.463289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457772 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 18:12:46.463289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457774 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 18:12:46.463289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457777 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 18:12:46.463289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457779 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 18:12:46.463289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457782 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 18:12:46.463289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457785 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 18:12:46.463289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457788 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 18:12:46.463289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457790 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 18:12:46.463289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457792 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 18:12:46.463289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457795 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 18:12:46.463289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457798 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 18:12:46.463289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457800 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 18:12:46.463289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457803 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 18:12:46.463289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457805 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 18:12:46.463289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457808 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 18:12:46.463289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457810 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 18:12:46.463289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457813 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 18:12:46.463289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457815 2572 feature_gate.go:328] unrecognized feature gate: Example Jun 08 18:12:46.463289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457818 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 18:12:46.463289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457820 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 18:12:46.463780 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457823 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 18:12:46.463780 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457826 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 18:12:46.463780 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457828 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 18:12:46.463780 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457831 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 18:12:46.463780 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457834 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 18:12:46.463780 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457836 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 18:12:46.463780 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457839 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 18:12:46.463780 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457841 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 18:12:46.463780 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457844 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 18:12:46.463780 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457847 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 18:12:46.463780 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457850 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 18:12:46.463780 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457852 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 18:12:46.463780 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457855 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 18:12:46.463780 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457857 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 18:12:46.463780 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457860 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 18:12:46.463780 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457862 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 18:12:46.463780 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457865 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 18:12:46.463780 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457868 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 18:12:46.463780 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457870 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 18:12:46.463780 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457873 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 18:12:46.464289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457875 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 18:12:46.464289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457878 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 18:12:46.464289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457881 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 18:12:46.464289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457884 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 18:12:46.464289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457887 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 18:12:46.464289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457891 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 18:12:46.464289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457893 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 18:12:46.464289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457896 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 18:12:46.464289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457899 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 18:12:46.464289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457901 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 18:12:46.464289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457904 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 18:12:46.464289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457906 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 18:12:46.464289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457909 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 18:12:46.464289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457911 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 18:12:46.464289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457914 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 18:12:46.464289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457917 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 18:12:46.464289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457920 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 18:12:46.464289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457922 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 18:12:46.464289 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457925 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 18:12:46.464743 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457928 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 18:12:46.464743 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457932 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 18:12:46.464743 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457935 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 18:12:46.464743 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457939 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 18:12:46.464743 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457942 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 18:12:46.464743 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457945 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 18:12:46.464743 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457947 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 18:12:46.464743 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457950 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 18:12:46.464743 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457952 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 18:12:46.464743 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457955 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 18:12:46.464743 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457974 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 18:12:46.464743 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457978 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 18:12:46.464743 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457983 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 18:12:46.464743 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457988 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 18:12:46.464743 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457991 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 18:12:46.464743 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457994 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 18:12:46.464743 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:46.457997 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 18:12:46.465208 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.458002 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 18:12:46.465208 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.458135 2572 server.go:962] "Client rotation is on, will bootstrap in background" Jun 08 18:12:46.465208 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.461624 2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 08 18:12:46.465208 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.462945 2572 server.go:1019] "Starting client certificate rotation" Jun 08 18:12:46.465208 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.463050 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 08 18:12:46.465208 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.463115 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 08 18:12:46.465918 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.465904 2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 08 18:12:46.467310 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.467283 2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 08 18:12:46.476025 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.475855 2572 log.go:25] "Validated CRI v1 runtime API" Jun 08 18:12:46.480249 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.480232 2572 log.go:25] "Validated CRI v1 image API" Jun 08 18:12:46.480980 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.480950 2572 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 08 18:12:46.483325 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.483298 2572 fs.go:135] Filesystem UUIDs: map[167fbf9c-7e98-4e1a-ab36-54ae9c99cd20:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c50ba04d-17d3-4b7e-a3b4-032c01904aa9:/dev/nvme0n1p3] Jun 08 18:12:46.483379 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.483326 2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 08 18:12:46.489184 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.489064 2572 manager.go:216] Machine: {Timestamp:2026-06-08 18:12:46.487572005 +0000 UTC m=+0.342078818 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3066459 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ee6d1eb8e93ec2ec888ec4cbadac0 SystemUUID:ec2ee6d1-eb8e-93ec-2ec8-88ec4cbadac0 BootID:43fbe1ca-a997-4328-8f1f-a890ed163ff1 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fc:34:84:59:bf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fc:34:84:59:bf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:99:e5:3c:8c:02 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 08 18:12:46.489184 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.489174 2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 08 18:12:46.489312 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.489263 2572 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 08 18:12:46.491581 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.491564 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 08 18:12:46.492708 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.492680 2572 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 08 18:12:46.492937 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.492712 2572 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-194.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 08 18:12:46.494108 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.494097 2572 topology_manager.go:138] "Creating topology manager with none policy" Jun 08 18:12:46.494146 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.494112 2572 container_manager_linux.go:309] "Creating device plugin manager" Jun 08 18:12:46.494146 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.494120 2572 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 08 18:12:46.494728 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.494718 2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 08 18:12:46.495623 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.495612 2572 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 08 18:12:46.498295 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.498284 2572 state_mem.go:36] "Initialized new in-memory state store" Jun 08 18:12:46.498419 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.498409 2572 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 08 18:12:46.498449 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.498441 2572 kubelet.go:486] "Attempting to sync node with API server" Jun 08 18:12:46.498481 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.498457 2572 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 08 18:12:46.498481 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.498480 2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 08 18:12:46.498534 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.498495 2572 kubelet.go:398] "Adding apiserver pod source" Jun 08 18:12:46.498534 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.498506 2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 08 18:12:46.498943 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.498930 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 08 18:12:46.499019 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.498948 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 08 18:12:46.501280 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.501263 2572 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 08 18:12:46.503039 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.503025 2572 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 08 18:12:46.503104 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.503055 2572 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 08 18:12:46.504066 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.504054 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 08 18:12:46.504116 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.504069 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 08 18:12:46.504116 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.504075 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 08 18:12:46.504116 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.504080 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 08 18:12:46.504116 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.504089 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 08 18:12:46.504116 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.504106 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 08 18:12:46.504116 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.504115 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 08 18:12:46.504289 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.504121 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 08 18:12:46.504289 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.504127 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 08 18:12:46.504289 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.504133 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 08 18:12:46.504289 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.504145 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 08 18:12:46.504289 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.504150 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 08 18:12:46.505569 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.505555 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 08 18:12:46.505569 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.505565 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 08 18:12:46.508702 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.508684 2572 server.go:1268] "Started kubelet" Jun 08 18:12:46.508782 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:46.508762 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-194.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 08 18:12:46.508849 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.508793 2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-194.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 08 18:12:46.508905 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.508857 2572 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 08 18:12:46.509033 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:46.509000 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 08 18:12:46.509157 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.509047 2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 08 18:12:46.509157 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.509107 2572 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 08 18:12:46.511945 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.511920 2572 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 08 18:12:46.512168 ip-10-0-138-194 systemd[1]: Started Kubernetes Kubelet. Jun 08 18:12:46.514430 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.513926 2572 server.go:310] "Adding debug handlers to kubelet server" Jun 08 18:12:46.515730 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.515707 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xtxwb" Jun 08 18:12:46.518281 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:46.517325 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-194.ec2.internal.18b72da581312539 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-194.ec2.internal,UID:ip-10-0-138-194.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-194.ec2.internal,},FirstTimestamp:2026-06-08 18:12:46.508655929 +0000 UTC m=+0.363162718,LastTimestamp:2026-06-08 18:12:46.508655929 +0000 UTC m=+0.363162718,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-194.ec2.internal,}" Jun 08 18:12:46.519706 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.519690 2572 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 08 18:12:46.519786 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.519687 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 08 18:12:46.519833 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:46.519789 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-194.ec2.internal\" not found" Jun 08 18:12:46.520256 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.520236 2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 08 18:12:46.520256 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.520251 2572 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 08 18:12:46.520382 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.520265 2572 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 08 18:12:46.520446 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.520402 2572 reconstruct.go:97] "Volume reconstruction finished" Jun 08 18:12:46.520446 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.520410 2572 reconciler.go:29] "Reconciler: start to sync state" Jun 08 18:12:46.520541 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:46.520487 2572 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 08 18:12:46.520614 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.520599 2572 factory.go:55] Registering systemd factory Jun 08 18:12:46.520661 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.520653 2572 factory.go:223] Registration of the systemd container factory successfully Jun 08 18:12:46.520901 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.520887 2572 factory.go:153] Registering CRI-O factory Jun 08 18:12:46.520994 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.520904 2572 factory.go:223] Registration of the crio container factory successfully Jun 08 18:12:46.520994 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.520980 2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 08 18:12:46.521098 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.521008 2572 factory.go:103] Registering Raw factory Jun 08 18:12:46.521098 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.521024 2572 manager.go:1196] Started watching for new ooms in manager Jun 08 18:12:46.521856 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.521842 2572 manager.go:349] Starting recovery of all containers Jun 08 18:12:46.523535 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.523507 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xtxwb" Jun 08 18:12:46.530476 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.530458 2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 18:12:46.530768 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.530754 2572 manager.go:354] Recovery completed Jun 08 18:12:46.534019 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:46.533999 2572 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-194.ec2.internal\" not found" node="ip-10-0-138-194.ec2.internal" Jun 08 18:12:46.535333 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.535319 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 18:12:46.537420 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.537407 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-194.ec2.internal" event="NodeHasSufficientMemory" Jun 08 18:12:46.537470 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.537434 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-194.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 18:12:46.537470 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.537444 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-194.ec2.internal" event="NodeHasSufficientPID" Jun 08 18:12:46.538907 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.538891 2572 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 08 18:12:46.538907 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.538905 2572 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 08 18:12:46.539093 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.538925 2572 state_mem.go:36] "Initialized new in-memory state store" Jun 08 18:12:46.539093 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.538897 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 18:12:46.539801 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.539784 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-194.ec2.internal" event="NodeHasSufficientMemory" Jun 08 18:12:46.539851 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.539820 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-194.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 18:12:46.539851 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.539835 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-194.ec2.internal" event="NodeHasSufficientPID" Jun 08 18:12:46.540286 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.540271 2572 policy_none.go:49] "None policy: Start" Jun 08 18:12:46.540348 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.540293 2572 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 08 18:12:46.540348 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.540308 2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 08 18:12:46.541292 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.541281 2572 policy_none.go:47] "Start" Jun 08 18:12:46.576584 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.576565 2572 manager.go:337] "Starting Device Plugin manager" Jun 08 18:12:46.605180 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:46.576606 2572 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 08 18:12:46.605180 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.576619 2572 server.go:87] "Starting device plugin registration server" Jun 08 18:12:46.605180 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.576879 2572 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 08 18:12:46.605180 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.576891 2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 08 18:12:46.605180 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.577038 2572 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 08 18:12:46.605180 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.577123 2572 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 08 18:12:46.605180 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.577132 2572 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 08 18:12:46.605180 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:46.577888 2572 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 08 18:12:46.605180 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:46.577941 2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-194.ec2.internal\" not found" Jun 08 18:12:46.605180 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.584750 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 08 18:12:46.605180 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.585917 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 08 18:12:46.605180 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.585973 2572 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 08 18:12:46.605180 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.585997 2572 kubelet.go:2448] "Starting kubelet main sync loop" Jun 08 18:12:46.605180 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:46.586083 2572 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 08 18:12:46.605180 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.588335 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 18:12:46.677363 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.677337 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 18:12:46.679800 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.679778 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-194.ec2.internal" event="NodeHasSufficientMemory" Jun 08 18:12:46.679906 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.679812 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-194.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 18:12:46.679906 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.679826 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-194.ec2.internal" event="NodeHasSufficientPID" Jun 08 18:12:46.679906 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.679851 2572 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-194.ec2.internal" Jun 08 18:12:46.686501 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.686483 2572 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-194.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-194.ec2.internal"] Jun 08 18:12:46.686563 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.686546 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 18:12:46.688004 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.687938 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-194.ec2.internal" event="NodeHasSufficientMemory" Jun 08 18:12:46.688004 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.687983 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-194.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 18:12:46.688004 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.687995 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-194.ec2.internal" event="NodeHasSufficientPID" Jun 08 18:12:46.688792 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.688776 2572 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-194.ec2.internal" Jun 08 18:12:46.688843 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:46.688806 2572 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-194.ec2.internal\": node \"ip-10-0-138-194.ec2.internal\" not found" Jun 08 18:12:46.689243 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.689231 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 18:12:46.689353 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.689340 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-194.ec2.internal" Jun 08 18:12:46.689408 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.689375 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 18:12:46.689901 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.689886 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-194.ec2.internal" event="NodeHasSufficientMemory" Jun 08 18:12:46.689978 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.689911 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-194.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 18:12:46.689978 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.689925 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-194.ec2.internal" event="NodeHasSufficientPID" Jun 08 18:12:46.690049 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.689981 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-194.ec2.internal" event="NodeHasSufficientMemory" Jun 08 18:12:46.690049 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.690009 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-194.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 18:12:46.690049 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.690021 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-194.ec2.internal" event="NodeHasSufficientPID" Jun 08 18:12:46.691211 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.691195 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-194.ec2.internal" Jun 08 18:12:46.691299 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.691220 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 18:12:46.691871 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.691857 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-194.ec2.internal" event="NodeHasSufficientMemory" Jun 08 18:12:46.691947 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.691882 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-194.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 18:12:46.691947 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.691898 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-194.ec2.internal" event="NodeHasSufficientPID" Jun 08 18:12:46.708604 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:46.708579 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-194.ec2.internal\" not found" node="ip-10-0-138-194.ec2.internal" Jun 08 18:12:46.712685 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:46.712668 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-194.ec2.internal\" not found" node="ip-10-0-138-194.ec2.internal" Jun 08 18:12:46.726319 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:46.726298 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-194.ec2.internal\" not found" Jun 08 18:12:46.821905 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.821867 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/58e732479eee24036ab8177dc98f421d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-194.ec2.internal\" (UID: \"58e732479eee24036ab8177dc98f421d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-194.ec2.internal" Jun 08 18:12:46.822020 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.821920 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58e732479eee24036ab8177dc98f421d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-194.ec2.internal\" (UID: \"58e732479eee24036ab8177dc98f421d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-194.ec2.internal" Jun 08 18:12:46.822020 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.821945 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d033c424e9151dca6b0d6a6e8144d8d3-config\") pod \"kube-apiserver-proxy-ip-10-0-138-194.ec2.internal\" (UID: \"d033c424e9151dca6b0d6a6e8144d8d3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-194.ec2.internal" Jun 08 18:12:46.826409 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:46.826387 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-194.ec2.internal\" not found" Jun 08 18:12:46.922786 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.922760 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/58e732479eee24036ab8177dc98f421d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-194.ec2.internal\" (UID: \"58e732479eee24036ab8177dc98f421d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-194.ec2.internal" Jun 08 18:12:46.922843 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.922795 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58e732479eee24036ab8177dc98f421d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-194.ec2.internal\" (UID: \"58e732479eee24036ab8177dc98f421d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-194.ec2.internal" Jun 08 18:12:46.922843 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.922832 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d033c424e9151dca6b0d6a6e8144d8d3-config\") pod \"kube-apiserver-proxy-ip-10-0-138-194.ec2.internal\" (UID: \"d033c424e9151dca6b0d6a6e8144d8d3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-194.ec2.internal" Jun 08 18:12:46.922904 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.922871 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d033c424e9151dca6b0d6a6e8144d8d3-config\") pod \"kube-apiserver-proxy-ip-10-0-138-194.ec2.internal\" (UID: \"d033c424e9151dca6b0d6a6e8144d8d3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-194.ec2.internal" Jun 08 18:12:46.922904 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.922874 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58e732479eee24036ab8177dc98f421d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-194.ec2.internal\" (UID: \"58e732479eee24036ab8177dc98f421d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-194.ec2.internal" Jun 08 18:12:46.922983 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:46.922913 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/58e732479eee24036ab8177dc98f421d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-194.ec2.internal\" (UID: \"58e732479eee24036ab8177dc98f421d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-194.ec2.internal" Jun 08 18:12:46.926849 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:46.926835 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-194.ec2.internal\" not found" Jun 08 18:12:47.012389 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:47.012287 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-194.ec2.internal" Jun 08 18:12:47.016725 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:47.016709 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-194.ec2.internal" Jun 08 18:12:47.027572 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:47.027550 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-194.ec2.internal\" not found" Jun 08 18:12:47.128072 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:47.128032 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-194.ec2.internal\" not found" Jun 08 18:12:47.228637 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:47.228602 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-194.ec2.internal\" not found" Jun 08 18:12:47.329154 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:47.329082 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-194.ec2.internal\" not found" Jun 08 18:12:47.429650 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:47.429620 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-194.ec2.internal\" not found" Jun 08 18:12:47.462841 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:47.462810 2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 08 18:12:47.463456 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:47.462984 2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 08 18:12:47.463456 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:47.463043 2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 08 18:12:47.525792 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:47.525752 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-07 18:07:46 +0000 UTC" deadline="2028-01-27 08:28:52.512531384 +0000 UTC" Jun 08 18:12:47.525792 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:47.525789 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14342h16m4.986747113s" Jun 08 18:12:47.529000 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:47.528954 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd033c424e9151dca6b0d6a6e8144d8d3.slice/crio-24f5153cffc2e07f2eef7ed01aa298a039d6f1d5fb48d831c7dc351dde5cf684 WatchSource:0}: Error finding container 24f5153cffc2e07f2eef7ed01aa298a039d6f1d5fb48d831c7dc351dde5cf684: Status 404 returned error can't find the container with id 24f5153cffc2e07f2eef7ed01aa298a039d6f1d5fb48d831c7dc351dde5cf684 Jun 08 18:12:47.530463 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:47.529739 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58e732479eee24036ab8177dc98f421d.slice/crio-131663cd17bc1a4a0f86f742436001f8dc33b4cc71036ab55ee73f56ea609c31 WatchSource:0}: Error finding container 131663cd17bc1a4a0f86f742436001f8dc33b4cc71036ab55ee73f56ea609c31: Status 404 returned error can't find the container with id 131663cd17bc1a4a0f86f742436001f8dc33b4cc71036ab55ee73f56ea609c31 Jun 08 18:12:47.530463 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:47.529780 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-194.ec2.internal\" not found" Jun 08 18:12:47.534862 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:47.534843 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 18:12:47.589420 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:47.589311 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-194.ec2.internal" event={"ID":"58e732479eee24036ab8177dc98f421d","Type":"ContainerStarted","Data":"131663cd17bc1a4a0f86f742436001f8dc33b4cc71036ab55ee73f56ea609c31"} Jun 08 18:12:47.590368 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:47.590345 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-194.ec2.internal" event={"ID":"d033c424e9151dca6b0d6a6e8144d8d3","Type":"ContainerStarted","Data":"24f5153cffc2e07f2eef7ed01aa298a039d6f1d5fb48d831c7dc351dde5cf684"} Jun 08 18:12:47.630892 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:47.630841 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-194.ec2.internal\" not found" Jun 08 18:12:47.731366 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:47.731329 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-194.ec2.internal\" not found" Jun 08 18:12:47.785392 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:47.785353 2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 18:12:47.796656 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:47.796614 2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 18:12:47.820437 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:47.820406 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-194.ec2.internal" Jun 08 18:12:47.830974 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:47.830938 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 08 18:12:47.832087 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:47.832073 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-194.ec2.internal" Jun 08 18:12:47.837884 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:47.837867 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 08 18:12:48.499702 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.499668 2572 apiserver.go:52] "Watching apiserver" Jun 08 18:12:48.505648 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.505623 2572 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 08 18:12:48.506128 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.506100 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-z77dq","openshift-multus/multus-additional-cni-plugins-ztjlr","openshift-multus/multus-qzwb8","openshift-ovn-kubernetes/ovnkube-node-jpwxq","kube-system/global-pull-secret-syncer-s8pt8","kube-system/konnectivity-agent-p892x","kube-system/kube-apiserver-proxy-ip-10-0-138-194.ec2.internal","openshift-image-registry/node-ca-bvg6f","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-194.ec2.internal","openshift-multus/network-metrics-daemon-jjs4l","openshift-network-diagnostics/network-check-target-kznkg","openshift-network-operator/iptables-alerter-pvn2c","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc","openshift-cluster-node-tuning-operator/tuned-4l6bl"] Jun 08 18:12:48.507899 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.507876 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:12:48.508005 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:48.507984 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjs4l" podUID="086390f2-8eb2-4903-a324-bf54451c435a" Jun 08 18:12:48.509100 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.509072 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" Jun 08 18:12:48.510478 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.510298 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.511214 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.511192 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 08 18:12:48.511309 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.511216 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xqdmg\"" Jun 08 18:12:48.511370 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.511327 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 08 18:12:48.511425 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.511404 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 08 18:12:48.511706 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.511674 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.512868 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.512784 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:12:48.512868 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:48.512845 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s8pt8" podUID="eca9e438-ae5a-4645-ab87-b8d44d7d35af" Jun 08 18:12:48.513043 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.512933 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 08 18:12:48.513097 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.513051 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 08 18:12:48.513097 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.513054 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 08 18:12:48.513268 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.513217 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 08 18:12:48.514205 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.514185 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p892x" Jun 08 18:12:48.515399 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.515370 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.516823 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.516257 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 08 18:12:48.516823 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.516294 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 08 18:12:48.516823 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.516344 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hdd2c\"" Jun 08 18:12:48.516823 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.516464 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 08 18:12:48.516823 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.516540 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 08 18:12:48.516823 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.516573 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bvg6f" Jun 08 18:12:48.516823 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.516653 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 08 18:12:48.516823 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.516721 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7bflq\"" Jun 08 18:12:48.517237 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.516944 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 08 18:12:48.518101 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.518072 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.518773 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.518750 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 08 18:12:48.519118 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.519100 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 08 18:12:48.519281 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.519261 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7f4j7\"" Jun 08 18:12:48.520300 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.520278 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 08 18:12:48.520377 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.520321 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 08 18:12:48.520775 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.520754 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 08 18:12:48.521480 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.521458 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z77dq" Jun 08 18:12:48.521576 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.521554 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-775vd\"" Jun 08 18:12:48.521878 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.521862 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qkbbl\"" Jun 08 18:12:48.521878 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.521871 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 08 18:12:48.522035 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.521885 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 08 18:12:48.523029 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.522166 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-n8d4f\"" Jun 08 18:12:48.523029 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.522204 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 08 18:12:48.523029 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.522216 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 08 18:12:48.523198 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.523061 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:12:48.523198 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:48.523143 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kznkg" podUID="c85176c2-6792-4e89-83bb-4e2056fbe983" Jun 08 18:12:48.524283 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.524263 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pvn2c" Jun 08 18:12:48.524776 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.524759 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9g7xl\"" Jun 08 18:12:48.524776 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.524771 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 08 18:12:48.525043 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.525026 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 08 18:12:48.525216 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.525203 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 08 18:12:48.526052 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.526036 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 08 18:12:48.526172 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.526040 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 08 18:12:48.526235 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.526100 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-srk9t\"" Jun 08 18:12:48.526838 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.526818 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 08 18:12:48.532808 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.532784 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/51fd7b9f-0366-434a-b567-33069764ed78-system-cni-dir\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.532912 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.532822 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-etc-sysctl-conf\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.532979 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.532906 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/51fd7b9f-0366-434a-b567-33069764ed78-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.532979 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.532932 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-host-run-k8s-cni-cncf-io\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.532979 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.532954 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-systemd-units\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.533128 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.532990 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-lib-modules\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.533128 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533013 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/78bb4fa4-4698-4288-a89c-fad4b5b5a825-etc-tuned\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.533128 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533035 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pwpk\" (UniqueName: \"kubernetes.io/projected/78bb4fa4-4698-4288-a89c-fad4b5b5a825-kube-api-access-8pwpk\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.533128 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533057 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78bb4fa4-4698-4288-a89c-fad4b5b5a825-tmp\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.533128 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533088 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmwmm\" (UniqueName: \"kubernetes.io/projected/086390f2-8eb2-4903-a324-bf54451c435a-kube-api-access-fmwmm\") pod \"network-metrics-daemon-jjs4l\" (UID: \"086390f2-8eb2-4903-a324-bf54451c435a\") " pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:12:48.533128 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533117 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1-etc-selinux\") pod \"aws-ebs-csi-driver-node-r8nlc\" (UID: \"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" Jun 08 18:12:48.533407 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533144 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret\") pod \"global-pull-secret-syncer-s8pt8\" (UID: \"eca9e438-ae5a-4645-ab87-b8d44d7d35af\") " pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:12:48.533407 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533171 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-host-kubelet\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.533407 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533197 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-log-socket\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.533407 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533234 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-host-cni-bin\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.533407 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533258 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-host-cni-netd\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.533407 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533273 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-host\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.533407 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533304 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/eca9e438-ae5a-4645-ab87-b8d44d7d35af-kubelet-config\") pod \"global-pull-secret-syncer-s8pt8\" (UID: \"eca9e438-ae5a-4645-ab87-b8d44d7d35af\") " pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:12:48.533407 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533317 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-host-var-lib-kubelet\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.533407 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533333 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-multus-conf-dir\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.533407 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533348 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/51fd7b9f-0366-434a-b567-33069764ed78-cni-binary-copy\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.533407 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533373 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/51fd7b9f-0366-434a-b567-33069764ed78-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.533407 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533395 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-cnibin\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.533981 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533416 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-host-run-netns\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.533981 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533438 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-host-slash\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.533981 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533460 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-host-run-netns\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.533981 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533482 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/51fd7b9f-0366-434a-b567-33069764ed78-os-release\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.533981 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533525 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2lsl\" (UniqueName: \"kubernetes.io/projected/51fd7b9f-0366-434a-b567-33069764ed78-kube-api-access-d2lsl\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.533981 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533560 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-system-cni-dir\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.533981 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533599 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-var-lib-openvswitch\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.533981 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533635 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-run-openvswitch\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.533981 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533671 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-etc-sysctl-d\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.533981 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533699 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/445f7d76-73b2-4d8c-aad9-9195be1bd5fa-serviceca\") pod \"node-ca-bvg6f\" (UID: \"445f7d76-73b2-4d8c-aad9-9195be1bd5fa\") " pod="openshift-image-registry/node-ca-bvg6f" Jun 08 18:12:48.533981 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533727 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs\") pod \"network-metrics-daemon-jjs4l\" (UID: \"086390f2-8eb2-4903-a324-bf54451c435a\") " pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:12:48.533981 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533765 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e27e2084-f792-4d85-9d63-99bb13520b73-agent-certs\") pod \"konnectivity-agent-p892x\" (UID: \"e27e2084-f792-4d85-9d63-99bb13520b73\") " pod="kube-system/konnectivity-agent-p892x" Jun 08 18:12:48.533981 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533787 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/51fd7b9f-0366-434a-b567-33069764ed78-cnibin\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.533981 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533810 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ad02886-8d99-4ad6-8103-a3eafea4af48-tmp-dir\") pod \"node-resolver-z77dq\" (UID: \"1ad02886-8d99-4ad6-8103-a3eafea4af48\") " pod="openshift-dns/node-resolver-z77dq" Jun 08 18:12:48.533981 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533831 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e2844f5-ca13-4cce-8fab-61322b119c50-cni-binary-copy\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.533981 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533853 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r98dv\" (UniqueName: \"kubernetes.io/projected/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-kube-api-access-r98dv\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.534837 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533877 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1-device-dir\") pod \"aws-ebs-csi-driver-node-r8nlc\" (UID: \"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" Jun 08 18:12:48.534837 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533898 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1-sys-fs\") pod \"aws-ebs-csi-driver-node-r8nlc\" (UID: \"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" Jun 08 18:12:48.534837 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533920 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nngzq\" (UniqueName: \"kubernetes.io/projected/367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1-kube-api-access-nngzq\") pod \"aws-ebs-csi-driver-node-r8nlc\" (UID: \"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" Jun 08 18:12:48.534837 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533939 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9e2844f5-ca13-4cce-8fab-61322b119c50-multus-daemon-config\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.534837 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533953 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-host-run-multus-certs\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.534837 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.533989 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-etc-openvswitch\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.534837 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534011 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-multus-cni-dir\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.534837 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534033 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-etc-kubernetes\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.534837 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534053 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-run-ovn\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.534837 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534109 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-etc-modprobe-d\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.534837 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534144 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ad02886-8d99-4ad6-8103-a3eafea4af48-hosts-file\") pod \"node-resolver-z77dq\" (UID: \"1ad02886-8d99-4ad6-8103-a3eafea4af48\") " pod="openshift-dns/node-resolver-z77dq" Jun 08 18:12:48.534837 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534169 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nphgz\" (UniqueName: \"kubernetes.io/projected/1ad02886-8d99-4ad6-8103-a3eafea4af48-kube-api-access-nphgz\") pod \"node-resolver-z77dq\" (UID: \"1ad02886-8d99-4ad6-8103-a3eafea4af48\") " pod="openshift-dns/node-resolver-z77dq" Jun 08 18:12:48.534837 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534193 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-host-var-lib-cni-multus\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.534837 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534217 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwbtl\" (UniqueName: \"kubernetes.io/projected/9e2844f5-ca13-4cce-8fab-61322b119c50-kube-api-access-rwbtl\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.534837 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534238 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-sys\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.534837 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534273 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-var-lib-kubelet\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.535643 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534302 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r8nlc\" (UID: \"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" Jun 08 18:12:48.535643 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534331 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-run-systemd\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.535643 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534356 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-host-run-ovn-kubernetes\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.535643 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534379 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1-socket-dir\") pod \"aws-ebs-csi-driver-node-r8nlc\" (UID: \"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" Jun 08 18:12:48.535643 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534404 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-os-release\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.535643 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534426 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-ovnkube-config\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.535643 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534450 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-etc-sysconfig\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.535643 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534475 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-multus-socket-dir-parent\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.535643 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534507 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-node-log\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.535643 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534530 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-ovn-node-metrics-cert\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.535643 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534562 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/445f7d76-73b2-4d8c-aad9-9195be1bd5fa-host\") pod \"node-ca-bvg6f\" (UID: \"445f7d76-73b2-4d8c-aad9-9195be1bd5fa\") " pod="openshift-image-registry/node-ca-bvg6f" Jun 08 18:12:48.535643 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534583 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85gtf\" (UniqueName: \"kubernetes.io/projected/445f7d76-73b2-4d8c-aad9-9195be1bd5fa-kube-api-access-85gtf\") pod \"node-ca-bvg6f\" (UID: \"445f7d76-73b2-4d8c-aad9-9195be1bd5fa\") " pod="openshift-image-registry/node-ca-bvg6f" Jun 08 18:12:48.535643 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534617 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/eca9e438-ae5a-4645-ab87-b8d44d7d35af-dbus\") pod \"global-pull-secret-syncer-s8pt8\" (UID: \"eca9e438-ae5a-4645-ab87-b8d44d7d35af\") " pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:12:48.535643 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534636 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e27e2084-f792-4d85-9d63-99bb13520b73-konnectivity-ca\") pod \"konnectivity-agent-p892x\" (UID: \"e27e2084-f792-4d85-9d63-99bb13520b73\") " pod="kube-system/konnectivity-agent-p892x" Jun 08 18:12:48.535643 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534660 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/51fd7b9f-0366-434a-b567-33069764ed78-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.535643 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534686 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-host-var-lib-cni-bin\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.536579 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534709 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.536579 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534732 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-env-overrides\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.536579 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534754 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-run\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.536579 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534778 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-ovnkube-script-lib\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.536579 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534800 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-etc-kubernetes\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.536579 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534822 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-etc-systemd\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.536579 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534842 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1-registration-dir\") pod \"aws-ebs-csi-driver-node-r8nlc\" (UID: \"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" Jun 08 18:12:48.536579 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.534860 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-hostroot\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.539132 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.539108 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 08 18:12:48.559487 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.559468 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-trsmf" Jun 08 18:12:48.568558 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.568538 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-trsmf" Jun 08 18:12:48.621426 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.621394 2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 08 18:12:48.635089 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.635054 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-os-release\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.635265 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.635095 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-ovnkube-config\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.635265 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.635123 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-etc-sysconfig\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.635265 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.635144 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-multus-socket-dir-parent\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.635265 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.635165 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-node-log\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.635265 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.635186 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-ovn-node-metrics-cert\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.635265 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.635207 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/445f7d76-73b2-4d8c-aad9-9195be1bd5fa-host\") pod \"node-ca-bvg6f\" (UID: \"445f7d76-73b2-4d8c-aad9-9195be1bd5fa\") " pod="openshift-image-registry/node-ca-bvg6f" Jun 08 18:12:48.635265 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.635228 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85gtf\" (UniqueName: \"kubernetes.io/projected/445f7d76-73b2-4d8c-aad9-9195be1bd5fa-kube-api-access-85gtf\") pod \"node-ca-bvg6f\" (UID: \"445f7d76-73b2-4d8c-aad9-9195be1bd5fa\") " pod="openshift-image-registry/node-ca-bvg6f" Jun 08 18:12:48.635265 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.635248 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/eca9e438-ae5a-4645-ab87-b8d44d7d35af-dbus\") pod \"global-pull-secret-syncer-s8pt8\" (UID: \"eca9e438-ae5a-4645-ab87-b8d44d7d35af\") " pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:12:48.635558 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.635269 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e27e2084-f792-4d85-9d63-99bb13520b73-konnectivity-ca\") pod \"konnectivity-agent-p892x\" (UID: \"e27e2084-f792-4d85-9d63-99bb13520b73\") " pod="kube-system/konnectivity-agent-p892x" Jun 08 18:12:48.635558 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.635292 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/51fd7b9f-0366-434a-b567-33069764ed78-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.635558 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.635316 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-host-var-lib-cni-bin\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.636015 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.635984 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/445f7d76-73b2-4d8c-aad9-9195be1bd5fa-host\") pod \"node-ca-bvg6f\" (UID: \"445f7d76-73b2-4d8c-aad9-9195be1bd5fa\") " pod="openshift-image-registry/node-ca-bvg6f" Jun 08 18:12:48.636015 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.636002 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.636203 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.636020 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-etc-sysconfig\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.636203 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.636058 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-env-overrides\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.637140 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.637084 2572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 08 18:12:48.637323 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.637263 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-env-overrides\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.637395 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.637347 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-os-release\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.637857 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.637833 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e27e2084-f792-4d85-9d63-99bb13520b73-konnectivity-ca\") pod \"konnectivity-agent-p892x\" (UID: \"e27e2084-f792-4d85-9d63-99bb13520b73\") " pod="kube-system/konnectivity-agent-p892x" Jun 08 18:12:48.637981 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.637853 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/eca9e438-ae5a-4645-ab87-b8d44d7d35af-dbus\") pod \"global-pull-secret-syncer-s8pt8\" (UID: \"eca9e438-ae5a-4645-ab87-b8d44d7d35af\") " pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:12:48.638046 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638001 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.638101 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638045 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-run\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.638101 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638077 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-ovnkube-script-lib\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.638190 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638107 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-etc-kubernetes\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.638190 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638132 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-etc-systemd\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.638190 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638161 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1-registration-dir\") pod \"aws-ebs-csi-driver-node-r8nlc\" (UID: \"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" Jun 08 18:12:48.638351 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638193 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-hostroot\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.638351 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638223 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/51fd7b9f-0366-434a-b567-33069764ed78-system-cni-dir\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.638351 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638249 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-etc-sysctl-conf\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.638351 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638277 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82m2r\" (UniqueName: \"kubernetes.io/projected/0f417f4b-e7ad-4076-bd6f-039a41d12fe9-kube-api-access-82m2r\") pod \"iptables-alerter-pvn2c\" (UID: \"0f417f4b-e7ad-4076-bd6f-039a41d12fe9\") " pod="openshift-network-operator/iptables-alerter-pvn2c" Jun 08 18:12:48.638351 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638309 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9986\" (UniqueName: \"kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986\") pod \"network-check-target-kznkg\" (UID: \"c85176c2-6792-4e89-83bb-4e2056fbe983\") " pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:12:48.638351 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638343 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/51fd7b9f-0366-434a-b567-33069764ed78-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.638607 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638374 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-host-run-k8s-cni-cncf-io\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.638607 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638402 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-systemd-units\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.638607 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638429 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-lib-modules\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.638607 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638429 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/51fd7b9f-0366-434a-b567-33069764ed78-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.638607 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638455 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/78bb4fa4-4698-4288-a89c-fad4b5b5a825-etc-tuned\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.638607 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638516 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/51fd7b9f-0366-434a-b567-33069764ed78-system-cni-dir\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.638910 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638646 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-etc-sysctl-conf\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.638910 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638655 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/51fd7b9f-0366-434a-b567-33069764ed78-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.638910 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638691 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-host-var-lib-cni-bin\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.638910 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638705 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-systemd-units\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.638910 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638730 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-host-run-k8s-cni-cncf-io\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.638910 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638789 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-multus-socket-dir-parent\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.638910 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638807 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-lib-modules\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.638910 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.638830 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-run\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.639293 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.639045 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-hostroot\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.639293 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.639120 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-etc-kubernetes\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.639293 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.639184 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-etc-systemd\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.639293 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.639253 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1-registration-dir\") pod \"aws-ebs-csi-driver-node-r8nlc\" (UID: \"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" Jun 08 18:12:48.639463 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.639349 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-node-log\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.639463 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.639423 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pwpk\" (UniqueName: \"kubernetes.io/projected/78bb4fa4-4698-4288-a89c-fad4b5b5a825-kube-api-access-8pwpk\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.639556 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.639463 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0f417f4b-e7ad-4076-bd6f-039a41d12fe9-host-slash\") pod \"iptables-alerter-pvn2c\" (UID: \"0f417f4b-e7ad-4076-bd6f-039a41d12fe9\") " pod="openshift-network-operator/iptables-alerter-pvn2c" Jun 08 18:12:48.639556 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.639502 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78bb4fa4-4698-4288-a89c-fad4b5b5a825-tmp\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.639556 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.639543 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmwmm\" (UniqueName: \"kubernetes.io/projected/086390f2-8eb2-4903-a324-bf54451c435a-kube-api-access-fmwmm\") pod \"network-metrics-daemon-jjs4l\" (UID: \"086390f2-8eb2-4903-a324-bf54451c435a\") " pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:12:48.639689 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.639580 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1-etc-selinux\") pod \"aws-ebs-csi-driver-node-r8nlc\" (UID: \"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" Jun 08 18:12:48.639689 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.639587 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-ovnkube-script-lib\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.639689 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.639612 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret\") pod \"global-pull-secret-syncer-s8pt8\" (UID: \"eca9e438-ae5a-4645-ab87-b8d44d7d35af\") " pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:12:48.639812 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.639699 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-host-kubelet\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.639812 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:48.639708 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 18:12:48.639812 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.639734 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-log-socket\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.639812 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.639773 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-host-cni-bin\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.639812 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:48.639792 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret podName:eca9e438-ae5a-4645-ab87-b8d44d7d35af nodeName:}" failed. No retries permitted until 2026-06-08 18:12:49.139770072 +0000 UTC m=+2.994276850 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret") pod "global-pull-secret-syncer-s8pt8" (UID: "eca9e438-ae5a-4645-ab87-b8d44d7d35af") : object "kube-system"/"original-pull-secret" not registered Jun 08 18:12:48.640065 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.639827 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-host-cni-bin\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.640065 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.639830 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-host-cni-netd\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.640065 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.639867 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-host\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.640065 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.639872 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-host-cni-netd\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.640065 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.639928 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/eca9e438-ae5a-4645-ab87-b8d44d7d35af-kubelet-config\") pod \"global-pull-secret-syncer-s8pt8\" (UID: \"eca9e438-ae5a-4645-ab87-b8d44d7d35af\") " pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:12:48.640065 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.639977 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0f417f4b-e7ad-4076-bd6f-039a41d12fe9-iptables-alerter-script\") pod \"iptables-alerter-pvn2c\" (UID: \"0f417f4b-e7ad-4076-bd6f-039a41d12fe9\") " pod="openshift-network-operator/iptables-alerter-pvn2c" Jun 08 18:12:48.640065 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.640014 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-host-var-lib-kubelet\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.640065 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.640065 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-multus-conf-dir\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.643072 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.640091 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/51fd7b9f-0366-434a-b567-33069764ed78-cni-binary-copy\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.643072 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.640131 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/51fd7b9f-0366-434a-b567-33069764ed78-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.643072 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.640257 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-cnibin\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.643072 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.640353 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-log-socket\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.643072 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.640475 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-host-var-lib-kubelet\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.643072 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.640614 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-multus-conf-dir\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.643072 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.640631 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-host-kubelet\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.643072 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.640674 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-host-run-netns\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.643072 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.640704 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-host-slash\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.643072 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.640724 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-cnibin\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.643072 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.640738 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-host-run-netns\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.643072 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.640827 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-host-run-netns\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.643072 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.640886 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-host-run-netns\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.643072 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.640886 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-host-slash\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.643072 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.640933 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-ovn-node-metrics-cert\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.643072 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.640992 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1-etc-selinux\") pod \"aws-ebs-csi-driver-node-r8nlc\" (UID: \"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" Jun 08 18:12:48.643072 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641038 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/eca9e438-ae5a-4645-ab87-b8d44d7d35af-kubelet-config\") pod \"global-pull-secret-syncer-s8pt8\" (UID: \"eca9e438-ae5a-4645-ab87-b8d44d7d35af\") " pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:12:48.643072 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641074 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/51fd7b9f-0366-434a-b567-33069764ed78-os-release\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.643882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641126 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/51fd7b9f-0366-434a-b567-33069764ed78-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.643882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641162 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-host\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.643882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641159 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2lsl\" (UniqueName: \"kubernetes.io/projected/51fd7b9f-0366-434a-b567-33069764ed78-kube-api-access-d2lsl\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.643882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641197 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/51fd7b9f-0366-434a-b567-33069764ed78-os-release\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.643882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641252 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-system-cni-dir\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.643882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641282 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-var-lib-openvswitch\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.643882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641306 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-run-openvswitch\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.643882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641334 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-etc-sysctl-d\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.643882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641369 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-var-lib-openvswitch\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.643882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641382 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/445f7d76-73b2-4d8c-aad9-9195be1bd5fa-serviceca\") pod \"node-ca-bvg6f\" (UID: \"445f7d76-73b2-4d8c-aad9-9195be1bd5fa\") " pod="openshift-image-registry/node-ca-bvg6f" Jun 08 18:12:48.643882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641417 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs\") pod \"network-metrics-daemon-jjs4l\" (UID: \"086390f2-8eb2-4903-a324-bf54451c435a\") " pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:12:48.643882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641438 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-run-openvswitch\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.643882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641451 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/51fd7b9f-0366-434a-b567-33069764ed78-cni-binary-copy\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.643882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641448 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e27e2084-f792-4d85-9d63-99bb13520b73-agent-certs\") pod \"konnectivity-agent-p892x\" (UID: \"e27e2084-f792-4d85-9d63-99bb13520b73\") " pod="kube-system/konnectivity-agent-p892x" Jun 08 18:12:48.643882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641502 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/51fd7b9f-0366-434a-b567-33069764ed78-cnibin\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.643882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641550 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ad02886-8d99-4ad6-8103-a3eafea4af48-tmp-dir\") pod \"node-resolver-z77dq\" (UID: \"1ad02886-8d99-4ad6-8103-a3eafea4af48\") " pod="openshift-dns/node-resolver-z77dq" Jun 08 18:12:48.643882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641562 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-etc-sysctl-d\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.643882 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:48.641582 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:12:48.644747 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641586 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e2844f5-ca13-4cce-8fab-61322b119c50-cni-binary-copy\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.644747 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641622 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r98dv\" (UniqueName: \"kubernetes.io/projected/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-kube-api-access-r98dv\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.644747 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641666 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1-device-dir\") pod \"aws-ebs-csi-driver-node-r8nlc\" (UID: \"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" Jun 08 18:12:48.644747 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641717 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1-sys-fs\") pod \"aws-ebs-csi-driver-node-r8nlc\" (UID: \"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" Jun 08 18:12:48.644747 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641753 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nngzq\" (UniqueName: \"kubernetes.io/projected/367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1-kube-api-access-nngzq\") pod \"aws-ebs-csi-driver-node-r8nlc\" (UID: \"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" Jun 08 18:12:48.644747 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641826 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9e2844f5-ca13-4cce-8fab-61322b119c50-multus-daemon-config\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.644747 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641855 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1-device-dir\") pod \"aws-ebs-csi-driver-node-r8nlc\" (UID: \"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" Jun 08 18:12:48.644747 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641909 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-host-run-multus-certs\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.644747 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641936 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-system-cni-dir\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.644747 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641987 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/445f7d76-73b2-4d8c-aad9-9195be1bd5fa-serviceca\") pod \"node-ca-bvg6f\" (UID: \"445f7d76-73b2-4d8c-aad9-9195be1bd5fa\") " pod="openshift-image-registry/node-ca-bvg6f" Jun 08 18:12:48.644747 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:48.642000 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs podName:086390f2-8eb2-4903-a324-bf54451c435a nodeName:}" failed. No retries permitted until 2026-06-08 18:12:49.141981132 +0000 UTC m=+2.996487909 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs") pod "network-metrics-daemon-jjs4l" (UID: "086390f2-8eb2-4903-a324-bf54451c435a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:12:48.644747 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642001 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ad02886-8d99-4ad6-8103-a3eafea4af48-tmp-dir\") pod \"node-resolver-z77dq\" (UID: \"1ad02886-8d99-4ad6-8103-a3eafea4af48\") " pod="openshift-dns/node-resolver-z77dq" Jun 08 18:12:48.644747 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.641941 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-etc-openvswitch\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.644747 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642055 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-multus-cni-dir\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.644747 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642083 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1-sys-fs\") pod \"aws-ebs-csi-driver-node-r8nlc\" (UID: \"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" Jun 08 18:12:48.644747 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642086 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-etc-kubernetes\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.644747 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642138 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-etc-kubernetes\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.645482 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642143 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-run-ovn\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.645482 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642017 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-etc-openvswitch\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.645482 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642177 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-etc-modprobe-d\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.645482 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642209 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ad02886-8d99-4ad6-8103-a3eafea4af48-hosts-file\") pod \"node-resolver-z77dq\" (UID: \"1ad02886-8d99-4ad6-8103-a3eafea4af48\") " pod="openshift-dns/node-resolver-z77dq" Jun 08 18:12:48.645482 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642216 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-multus-cni-dir\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.645482 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642276 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ad02886-8d99-4ad6-8103-a3eafea4af48-hosts-file\") pod \"node-resolver-z77dq\" (UID: \"1ad02886-8d99-4ad6-8103-a3eafea4af48\") " pod="openshift-dns/node-resolver-z77dq" Jun 08 18:12:48.645482 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642321 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nphgz\" (UniqueName: \"kubernetes.io/projected/1ad02886-8d99-4ad6-8103-a3eafea4af48-kube-api-access-nphgz\") pod \"node-resolver-z77dq\" (UID: \"1ad02886-8d99-4ad6-8103-a3eafea4af48\") " pod="openshift-dns/node-resolver-z77dq" Jun 08 18:12:48.645482 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642389 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-host-run-multus-certs\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.645482 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642387 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-host-var-lib-cni-multus\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.645482 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642429 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9e2844f5-ca13-4cce-8fab-61322b119c50-host-var-lib-cni-multus\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.645482 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642469 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e2844f5-ca13-4cce-8fab-61322b119c50-cni-binary-copy\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.645482 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642430 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwbtl\" (UniqueName: \"kubernetes.io/projected/9e2844f5-ca13-4cce-8fab-61322b119c50-kube-api-access-rwbtl\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.645482 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642545 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9e2844f5-ca13-4cce-8fab-61322b119c50-multus-daemon-config\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.645482 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642574 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-sys\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.645482 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642617 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-sys\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.645482 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642772 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-var-lib-kubelet\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.645482 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642814 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-var-lib-kubelet\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.645482 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642849 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r8nlc\" (UID: \"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" Jun 08 18:12:48.646188 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642892 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-run-systemd\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.646188 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642941 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-host-run-ovn-kubernetes\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.646188 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642949 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/78bb4fa4-4698-4288-a89c-fad4b5b5a825-etc-modprobe-d\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.646188 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642991 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1-socket-dir\") pod \"aws-ebs-csi-driver-node-r8nlc\" (UID: \"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" Jun 08 18:12:48.646188 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.643126 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1-socket-dir\") pod \"aws-ebs-csi-driver-node-r8nlc\" (UID: \"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" Jun 08 18:12:48.646188 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.643184 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r8nlc\" (UID: \"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" Jun 08 18:12:48.646188 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.642849 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-run-ovn\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.646188 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.643243 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-run-systemd\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.646188 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.643319 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-host-run-ovn-kubernetes\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.646188 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.643391 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/51fd7b9f-0366-434a-b567-33069764ed78-cnibin\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.646188 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.643741 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/78bb4fa4-4698-4288-a89c-fad4b5b5a825-etc-tuned\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.646188 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.644282 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78bb4fa4-4698-4288-a89c-fad4b5b5a825-tmp\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.646188 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.644683 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-ovnkube-config\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.646188 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.645605 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e27e2084-f792-4d85-9d63-99bb13520b73-agent-certs\") pod \"konnectivity-agent-p892x\" (UID: \"e27e2084-f792-4d85-9d63-99bb13520b73\") " pod="kube-system/konnectivity-agent-p892x" Jun 08 18:12:48.649239 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.649213 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pwpk\" (UniqueName: \"kubernetes.io/projected/78bb4fa4-4698-4288-a89c-fad4b5b5a825-kube-api-access-8pwpk\") pod \"tuned-4l6bl\" (UID: \"78bb4fa4-4698-4288-a89c-fad4b5b5a825\") " pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.649565 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.649546 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmwmm\" (UniqueName: \"kubernetes.io/projected/086390f2-8eb2-4903-a324-bf54451c435a-kube-api-access-fmwmm\") pod \"network-metrics-daemon-jjs4l\" (UID: \"086390f2-8eb2-4903-a324-bf54451c435a\") " pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:12:48.650475 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.650448 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nngzq\" (UniqueName: \"kubernetes.io/projected/367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1-kube-api-access-nngzq\") pod \"aws-ebs-csi-driver-node-r8nlc\" (UID: \"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" Jun 08 18:12:48.650822 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.650799 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2lsl\" (UniqueName: \"kubernetes.io/projected/51fd7b9f-0366-434a-b567-33069764ed78-kube-api-access-d2lsl\") pod \"multus-additional-cni-plugins-ztjlr\" (UID: \"51fd7b9f-0366-434a-b567-33069764ed78\") " pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.651647 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.651626 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r98dv\" (UniqueName: \"kubernetes.io/projected/ac833fb5-5b86-4ab4-aa93-2441a8a30ff1-kube-api-access-r98dv\") pod \"ovnkube-node-jpwxq\" (UID: \"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1\") " pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.654727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.654704 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwbtl\" (UniqueName: \"kubernetes.io/projected/9e2844f5-ca13-4cce-8fab-61322b119c50-kube-api-access-rwbtl\") pod \"multus-qzwb8\" (UID: \"9e2844f5-ca13-4cce-8fab-61322b119c50\") " pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.655427 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.655387 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85gtf\" (UniqueName: \"kubernetes.io/projected/445f7d76-73b2-4d8c-aad9-9195be1bd5fa-kube-api-access-85gtf\") pod \"node-ca-bvg6f\" (UID: \"445f7d76-73b2-4d8c-aad9-9195be1bd5fa\") " pod="openshift-image-registry/node-ca-bvg6f" Jun 08 18:12:48.657266 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.657245 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nphgz\" (UniqueName: \"kubernetes.io/projected/1ad02886-8d99-4ad6-8103-a3eafea4af48-kube-api-access-nphgz\") pod \"node-resolver-z77dq\" (UID: \"1ad02886-8d99-4ad6-8103-a3eafea4af48\") " pod="openshift-dns/node-resolver-z77dq" Jun 08 18:12:48.744088 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.744057 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82m2r\" (UniqueName: \"kubernetes.io/projected/0f417f4b-e7ad-4076-bd6f-039a41d12fe9-kube-api-access-82m2r\") pod \"iptables-alerter-pvn2c\" (UID: \"0f417f4b-e7ad-4076-bd6f-039a41d12fe9\") " pod="openshift-network-operator/iptables-alerter-pvn2c" Jun 08 18:12:48.744088 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.744091 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9986\" (UniqueName: \"kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986\") pod \"network-check-target-kznkg\" (UID: \"c85176c2-6792-4e89-83bb-4e2056fbe983\") " pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:12:48.744309 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.744111 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0f417f4b-e7ad-4076-bd6f-039a41d12fe9-host-slash\") pod \"iptables-alerter-pvn2c\" (UID: \"0f417f4b-e7ad-4076-bd6f-039a41d12fe9\") " pod="openshift-network-operator/iptables-alerter-pvn2c" Jun 08 18:12:48.744309 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.744167 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0f417f4b-e7ad-4076-bd6f-039a41d12fe9-host-slash\") pod \"iptables-alerter-pvn2c\" (UID: \"0f417f4b-e7ad-4076-bd6f-039a41d12fe9\") " pod="openshift-network-operator/iptables-alerter-pvn2c" Jun 08 18:12:48.744309 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.744248 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0f417f4b-e7ad-4076-bd6f-039a41d12fe9-iptables-alerter-script\") pod \"iptables-alerter-pvn2c\" (UID: \"0f417f4b-e7ad-4076-bd6f-039a41d12fe9\") " pod="openshift-network-operator/iptables-alerter-pvn2c" Jun 08 18:12:48.744717 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.744700 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0f417f4b-e7ad-4076-bd6f-039a41d12fe9-iptables-alerter-script\") pod \"iptables-alerter-pvn2c\" (UID: \"0f417f4b-e7ad-4076-bd6f-039a41d12fe9\") " pod="openshift-network-operator/iptables-alerter-pvn2c" Jun 08 18:12:48.752456 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:48.752394 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 18:12:48.752456 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:48.752422 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 18:12:48.752456 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:48.752436 2572 projected.go:196] Error preparing data for projected volume kube-api-access-f9986 for pod openshift-network-diagnostics/network-check-target-kznkg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:12:48.752629 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:48.752507 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986 podName:c85176c2-6792-4e89-83bb-4e2056fbe983 nodeName:}" failed. No retries permitted until 2026-06-08 18:12:49.25248774 +0000 UTC m=+3.106994541 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-f9986" (UniqueName: "kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986") pod "network-check-target-kznkg" (UID: "c85176c2-6792-4e89-83bb-4e2056fbe983") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:12:48.754504 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.754486 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82m2r\" (UniqueName: \"kubernetes.io/projected/0f417f4b-e7ad-4076-bd6f-039a41d12fe9-kube-api-access-82m2r\") pod \"iptables-alerter-pvn2c\" (UID: \"0f417f4b-e7ad-4076-bd6f-039a41d12fe9\") " pod="openshift-network-operator/iptables-alerter-pvn2c" Jun 08 18:12:48.822459 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.822418 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" Jun 08 18:12:48.830998 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.830955 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qzwb8" Jun 08 18:12:48.839200 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.839167 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:12:48.845067 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.845044 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p892x" Jun 08 18:12:48.852734 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.852706 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" Jun 08 18:12:48.860347 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.860329 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bvg6f" Jun 08 18:12:48.867949 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.867921 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ztjlr" Jun 08 18:12:48.873438 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.873419 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z77dq" Jun 08 18:12:48.879993 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.879974 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pvn2c" Jun 08 18:12:48.976816 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:48.976785 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 18:12:49.021644 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:49.021567 2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 18:12:49.147516 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:49.147482 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs\") pod \"network-metrics-daemon-jjs4l\" (UID: \"086390f2-8eb2-4903-a324-bf54451c435a\") " pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:12:49.147685 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:49.147547 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret\") pod \"global-pull-secret-syncer-s8pt8\" (UID: \"eca9e438-ae5a-4645-ab87-b8d44d7d35af\") " pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:12:49.147685 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:49.147659 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 18:12:49.147685 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:49.147665 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:12:49.147832 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:49.147719 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret podName:eca9e438-ae5a-4645-ab87-b8d44d7d35af nodeName:}" failed. No retries permitted until 2026-06-08 18:12:50.147701648 +0000 UTC m=+4.002208442 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret") pod "global-pull-secret-syncer-s8pt8" (UID: "eca9e438-ae5a-4645-ab87-b8d44d7d35af") : object "kube-system"/"original-pull-secret" not registered Jun 08 18:12:49.147832 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:49.147736 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs podName:086390f2-8eb2-4903-a324-bf54451c435a nodeName:}" failed. No retries permitted until 2026-06-08 18:12:50.147728804 +0000 UTC m=+4.002235580 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs") pod "network-metrics-daemon-jjs4l" (UID: "086390f2-8eb2-4903-a324-bf54451c435a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:12:49.348481 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:49.348387 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9986\" (UniqueName: \"kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986\") pod \"network-check-target-kznkg\" (UID: \"c85176c2-6792-4e89-83bb-4e2056fbe983\") " pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:12:49.348620 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:49.348569 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 18:12:49.348620 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:49.348587 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 18:12:49.348620 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:49.348596 2572 projected.go:196] Error preparing data for projected volume kube-api-access-f9986 for pod openshift-network-diagnostics/network-check-target-kznkg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:12:49.348762 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:49.348659 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986 podName:c85176c2-6792-4e89-83bb-4e2056fbe983 nodeName:}" failed. No retries permitted until 2026-06-08 18:12:50.348639659 +0000 UTC m=+4.203146459 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9986" (UniqueName: "kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986") pod "network-check-target-kznkg" (UID: "c85176c2-6792-4e89-83bb-4e2056fbe983") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:12:49.416110 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:49.416070 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod367102c3_3e6a_4c17_9d5d_a8e6a2df7fb1.slice/crio-9864394f8e40a22cc217e6655add2b9b1b013c19fc105f4b940544931e1db3d6 WatchSource:0}: Error finding container 9864394f8e40a22cc217e6655add2b9b1b013c19fc105f4b940544931e1db3d6: Status 404 returned error can't find the container with id 9864394f8e40a22cc217e6655add2b9b1b013c19fc105f4b940544931e1db3d6 Jun 08 18:12:49.417713 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:49.417682 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac833fb5_5b86_4ab4_aa93_2441a8a30ff1.slice/crio-f7ee03f48b128dd051746d488f287c532c7d2d708bdc5e7d6315cf3755b1f5cb WatchSource:0}: Error finding container f7ee03f48b128dd051746d488f287c532c7d2d708bdc5e7d6315cf3755b1f5cb: Status 404 returned error can't find the container with id f7ee03f48b128dd051746d488f287c532c7d2d708bdc5e7d6315cf3755b1f5cb Jun 08 18:12:49.419011 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:49.418942 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f417f4b_e7ad_4076_bd6f_039a41d12fe9.slice/crio-500a80249c3adf9ab6fb90de3ac9041d5eea0a3c3bf1f779efd4489bf82be0a7 WatchSource:0}: Error finding container 500a80249c3adf9ab6fb90de3ac9041d5eea0a3c3bf1f779efd4489bf82be0a7: Status 404 returned error can't find the container with id 500a80249c3adf9ab6fb90de3ac9041d5eea0a3c3bf1f779efd4489bf82be0a7 Jun 08 18:12:49.420219 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:49.419789 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ad02886_8d99_4ad6_8103_a3eafea4af48.slice/crio-a4c961b6de0e8ffb72f5b8542176d30193e65c19cfa64e8fa29c7bc0be670bb1 WatchSource:0}: Error finding container a4c961b6de0e8ffb72f5b8542176d30193e65c19cfa64e8fa29c7bc0be670bb1: Status 404 returned error can't find the container with id a4c961b6de0e8ffb72f5b8542176d30193e65c19cfa64e8fa29c7bc0be670bb1 Jun 08 18:12:49.425674 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:49.425568 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51fd7b9f_0366_434a_b567_33069764ed78.slice/crio-4038b9a7e46c58df785b1427da8480f8700d192099831eca13b81ae612e149dc WatchSource:0}: Error finding container 4038b9a7e46c58df785b1427da8480f8700d192099831eca13b81ae612e149dc: Status 404 returned error can't find the container with id 4038b9a7e46c58df785b1427da8480f8700d192099831eca13b81ae612e149dc Jun 08 18:12:49.426620 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:49.426593 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode27e2084_f792_4d85_9d63_99bb13520b73.slice/crio-c1df4c675a16eca7f0c3188fc745f33247de5aa9ae52f0d82edcc5eb29357f98 WatchSource:0}: Error finding container c1df4c675a16eca7f0c3188fc745f33247de5aa9ae52f0d82edcc5eb29357f98: Status 404 returned error can't find the container with id c1df4c675a16eca7f0c3188fc745f33247de5aa9ae52f0d82edcc5eb29357f98 Jun 08 18:12:49.428172 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:49.428146 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod445f7d76_73b2_4d8c_aad9_9195be1bd5fa.slice/crio-28aea64fb56fc01f6f07e09b2895574b4f6d735871ab57f25939292e509d26ed WatchSource:0}: Error finding container 28aea64fb56fc01f6f07e09b2895574b4f6d735871ab57f25939292e509d26ed: Status 404 returned error can't find the container with id 28aea64fb56fc01f6f07e09b2895574b4f6d735871ab57f25939292e509d26ed Jun 08 18:12:49.428482 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:12:49.428462 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e2844f5_ca13_4cce_8fab_61322b119c50.slice/crio-e5688056aba4634cc5515ba1732c1d22a3368b61327ade95199de4fa5acc86e2 WatchSource:0}: Error finding container e5688056aba4634cc5515ba1732c1d22a3368b61327ade95199de4fa5acc86e2: Status 404 returned error can't find the container with id e5688056aba4634cc5515ba1732c1d22a3368b61327ade95199de4fa5acc86e2 Jun 08 18:12:49.571332 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:49.571276 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-07 18:07:48 +0000 UTC" deadline="2028-02-18 08:48:51.739871322 +0000 UTC" Jun 08 18:12:49.571332 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:49.571326 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14870h36m2.168549071s" Jun 08 18:12:49.595255 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:49.595220 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztjlr" event={"ID":"51fd7b9f-0366-434a-b567-33069764ed78","Type":"ContainerStarted","Data":"4038b9a7e46c58df785b1427da8480f8700d192099831eca13b81ae612e149dc"} Jun 08 18:12:49.596206 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:49.596181 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z77dq" event={"ID":"1ad02886-8d99-4ad6-8103-a3eafea4af48","Type":"ContainerStarted","Data":"a4c961b6de0e8ffb72f5b8542176d30193e65c19cfa64e8fa29c7bc0be670bb1"} Jun 08 18:12:49.597377 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:49.597344 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pvn2c" event={"ID":"0f417f4b-e7ad-4076-bd6f-039a41d12fe9","Type":"ContainerStarted","Data":"500a80249c3adf9ab6fb90de3ac9041d5eea0a3c3bf1f779efd4489bf82be0a7"} Jun 08 18:12:49.598604 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:49.598519 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" event={"ID":"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1","Type":"ContainerStarted","Data":"f7ee03f48b128dd051746d488f287c532c7d2d708bdc5e7d6315cf3755b1f5cb"} Jun 08 18:12:49.600494 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:49.600464 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-194.ec2.internal" event={"ID":"d033c424e9151dca6b0d6a6e8144d8d3","Type":"ContainerStarted","Data":"de26c6345be65eeca083417f218a8ae56fb786ca6dc5528b7a717dc58b0db1c1"} Jun 08 18:12:49.601532 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:49.601508 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bvg6f" event={"ID":"445f7d76-73b2-4d8c-aad9-9195be1bd5fa","Type":"ContainerStarted","Data":"28aea64fb56fc01f6f07e09b2895574b4f6d735871ab57f25939292e509d26ed"} Jun 08 18:12:49.602702 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:49.602670 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" event={"ID":"78bb4fa4-4698-4288-a89c-fad4b5b5a825","Type":"ContainerStarted","Data":"3fc7128229cd6991e73a96f616675fca5dacc4ce8ea1a318cbd2876615137b7b"} Jun 08 18:12:49.603657 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:49.603636 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" event={"ID":"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1","Type":"ContainerStarted","Data":"9864394f8e40a22cc217e6655add2b9b1b013c19fc105f4b940544931e1db3d6"} Jun 08 18:12:49.604906 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:49.604879 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qzwb8" event={"ID":"9e2844f5-ca13-4cce-8fab-61322b119c50","Type":"ContainerStarted","Data":"e5688056aba4634cc5515ba1732c1d22a3368b61327ade95199de4fa5acc86e2"} Jun 08 18:12:49.607894 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:49.607872 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p892x" event={"ID":"e27e2084-f792-4d85-9d63-99bb13520b73","Type":"ContainerStarted","Data":"c1df4c675a16eca7f0c3188fc745f33247de5aa9ae52f0d82edcc5eb29357f98"} Jun 08 18:12:50.156305 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:50.156243 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs\") pod \"network-metrics-daemon-jjs4l\" (UID: \"086390f2-8eb2-4903-a324-bf54451c435a\") " pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:12:50.156526 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:50.156346 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret\") pod \"global-pull-secret-syncer-s8pt8\" (UID: \"eca9e438-ae5a-4645-ab87-b8d44d7d35af\") " pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:12:50.156526 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:50.156464 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 18:12:50.156526 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:50.156524 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret podName:eca9e438-ae5a-4645-ab87-b8d44d7d35af nodeName:}" failed. No retries permitted until 2026-06-08 18:12:52.15650733 +0000 UTC m=+6.011014119 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret") pod "global-pull-secret-syncer-s8pt8" (UID: "eca9e438-ae5a-4645-ab87-b8d44d7d35af") : object "kube-system"/"original-pull-secret" not registered Jun 08 18:12:50.156950 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:50.156923 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:12:50.157068 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:50.157001 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs podName:086390f2-8eb2-4903-a324-bf54451c435a nodeName:}" failed. No retries permitted until 2026-06-08 18:12:52.156984502 +0000 UTC m=+6.011491277 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs") pod "network-metrics-daemon-jjs4l" (UID: "086390f2-8eb2-4903-a324-bf54451c435a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:12:50.358402 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:50.358360 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9986\" (UniqueName: \"kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986\") pod \"network-check-target-kznkg\" (UID: \"c85176c2-6792-4e89-83bb-4e2056fbe983\") " pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:12:50.358601 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:50.358574 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 18:12:50.358662 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:50.358601 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 18:12:50.358662 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:50.358617 2572 projected.go:196] Error preparing data for projected volume kube-api-access-f9986 for pod openshift-network-diagnostics/network-check-target-kznkg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:12:50.358771 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:50.358675 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986 podName:c85176c2-6792-4e89-83bb-4e2056fbe983 nodeName:}" failed. No retries permitted until 2026-06-08 18:12:52.358656657 +0000 UTC m=+6.213163456 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9986" (UniqueName: "kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986") pod "network-check-target-kznkg" (UID: "c85176c2-6792-4e89-83bb-4e2056fbe983") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:12:50.572184 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:50.572058 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-07 18:07:48 +0000 UTC" deadline="2028-03-08 10:25:10.040847982 +0000 UTC" Jun 08 18:12:50.572184 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:50.572101 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15328h12m19.468752423s" Jun 08 18:12:50.589429 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:50.588638 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:12:50.589429 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:50.588764 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s8pt8" podUID="eca9e438-ae5a-4645-ab87-b8d44d7d35af" Jun 08 18:12:50.589429 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:50.589156 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:12:50.589429 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:50.589245 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kznkg" podUID="c85176c2-6792-4e89-83bb-4e2056fbe983" Jun 08 18:12:50.589429 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:50.589304 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:12:50.589429 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:50.589377 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjs4l" podUID="086390f2-8eb2-4903-a324-bf54451c435a" Jun 08 18:12:50.632358 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:50.632318 2572 generic.go:356] "Generic (PLEG): container finished" podID="58e732479eee24036ab8177dc98f421d" containerID="d79a8160e41d073da518796af28603e1d55708cd26638c769a4bde0191e18258" exitCode=0 Jun 08 18:12:50.633293 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:50.633261 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-194.ec2.internal" event={"ID":"58e732479eee24036ab8177dc98f421d","Type":"ContainerDied","Data":"d79a8160e41d073da518796af28603e1d55708cd26638c769a4bde0191e18258"} Jun 08 18:12:50.649241 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:50.649183 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-194.ec2.internal" podStartSLOduration=3.6491607999999998 podStartE2EDuration="3.6491608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:12:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:12:49.614354986 +0000 UTC m=+3.468861784" watchObservedRunningTime="2026-06-08 18:12:50.6491608 +0000 UTC m=+4.503667601" Jun 08 18:12:51.663293 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:51.663206 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-194.ec2.internal" event={"ID":"58e732479eee24036ab8177dc98f421d","Type":"ContainerStarted","Data":"21869242756e7c57a1be2ca6b9ddff732e4efbeac8f268a7c2a3bbfee476f389"} Jun 08 18:12:51.681374 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:51.680859 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-194.ec2.internal" podStartSLOduration=4.680838312 podStartE2EDuration="4.680838312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:12:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:12:51.680060174 +0000 UTC m=+5.534566973" watchObservedRunningTime="2026-06-08 18:12:51.680838312 +0000 UTC m=+5.535345111" Jun 08 18:12:52.176182 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:52.176142 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs\") pod \"network-metrics-daemon-jjs4l\" (UID: \"086390f2-8eb2-4903-a324-bf54451c435a\") " pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:12:52.176356 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:52.176267 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret\") pod \"global-pull-secret-syncer-s8pt8\" (UID: \"eca9e438-ae5a-4645-ab87-b8d44d7d35af\") " pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:12:52.176415 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:52.176396 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 18:12:52.176481 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:52.176460 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret podName:eca9e438-ae5a-4645-ab87-b8d44d7d35af nodeName:}" failed. No retries permitted until 2026-06-08 18:12:56.176442468 +0000 UTC m=+10.030949247 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret") pod "global-pull-secret-syncer-s8pt8" (UID: "eca9e438-ae5a-4645-ab87-b8d44d7d35af") : object "kube-system"/"original-pull-secret" not registered Jun 08 18:12:52.176910 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:52.176890 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:12:52.177016 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:52.176944 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs podName:086390f2-8eb2-4903-a324-bf54451c435a nodeName:}" failed. No retries permitted until 2026-06-08 18:12:56.176929041 +0000 UTC m=+10.031435818 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs") pod "network-metrics-daemon-jjs4l" (UID: "086390f2-8eb2-4903-a324-bf54451c435a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:12:52.377775 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:52.377739 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9986\" (UniqueName: \"kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986\") pod \"network-check-target-kznkg\" (UID: \"c85176c2-6792-4e89-83bb-4e2056fbe983\") " pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:12:52.377986 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:52.377929 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 18:12:52.377986 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:52.377975 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 18:12:52.378115 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:52.377991 2572 projected.go:196] Error preparing data for projected volume kube-api-access-f9986 for pod openshift-network-diagnostics/network-check-target-kznkg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:12:52.378115 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:52.378054 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986 podName:c85176c2-6792-4e89-83bb-4e2056fbe983 nodeName:}" failed. No retries permitted until 2026-06-08 18:12:56.378035267 +0000 UTC m=+10.232542067 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9986" (UniqueName: "kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986") pod "network-check-target-kznkg" (UID: "c85176c2-6792-4e89-83bb-4e2056fbe983") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:12:52.587301 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:52.586736 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:12:52.587301 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:52.586766 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:12:52.587301 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:52.586742 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:12:52.587301 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:52.586887 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjs4l" podUID="086390f2-8eb2-4903-a324-bf54451c435a" Jun 08 18:12:52.587301 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:52.586985 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s8pt8" podUID="eca9e438-ae5a-4645-ab87-b8d44d7d35af" Jun 08 18:12:52.587301 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:52.587144 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kznkg" podUID="c85176c2-6792-4e89-83bb-4e2056fbe983" Jun 08 18:12:54.586710 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:54.586677 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:12:54.587198 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:54.586677 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:12:54.587198 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:54.586833 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjs4l" podUID="086390f2-8eb2-4903-a324-bf54451c435a" Jun 08 18:12:54.587198 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:54.586692 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:12:54.587198 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:54.586924 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s8pt8" podUID="eca9e438-ae5a-4645-ab87-b8d44d7d35af" Jun 08 18:12:54.587198 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:54.587044 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kznkg" podUID="c85176c2-6792-4e89-83bb-4e2056fbe983" Jun 08 18:12:56.210440 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:56.210396 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret\") pod \"global-pull-secret-syncer-s8pt8\" (UID: \"eca9e438-ae5a-4645-ab87-b8d44d7d35af\") " pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:12:56.210888 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:56.210464 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs\") pod \"network-metrics-daemon-jjs4l\" (UID: \"086390f2-8eb2-4903-a324-bf54451c435a\") " pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:12:56.210888 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:56.210594 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:12:56.210888 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:56.210656 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs podName:086390f2-8eb2-4903-a324-bf54451c435a nodeName:}" failed. No retries permitted until 2026-06-08 18:13:04.210637377 +0000 UTC m=+18.065144166 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs") pod "network-metrics-daemon-jjs4l" (UID: "086390f2-8eb2-4903-a324-bf54451c435a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:12:56.211098 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:56.211079 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 18:12:56.211154 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:56.211129 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret podName:eca9e438-ae5a-4645-ab87-b8d44d7d35af nodeName:}" failed. No retries permitted until 2026-06-08 18:13:04.211114744 +0000 UTC m=+18.065621534 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret") pod "global-pull-secret-syncer-s8pt8" (UID: "eca9e438-ae5a-4645-ab87-b8d44d7d35af") : object "kube-system"/"original-pull-secret" not registered Jun 08 18:12:56.412875 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:56.412832 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9986\" (UniqueName: \"kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986\") pod \"network-check-target-kznkg\" (UID: \"c85176c2-6792-4e89-83bb-4e2056fbe983\") " pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:12:56.413073 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:56.413033 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 18:12:56.413073 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:56.413058 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 18:12:56.413073 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:56.413071 2572 projected.go:196] Error preparing data for projected volume kube-api-access-f9986 for pod openshift-network-diagnostics/network-check-target-kznkg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:12:56.413244 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:56.413130 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986 podName:c85176c2-6792-4e89-83bb-4e2056fbe983 nodeName:}" failed. No retries permitted until 2026-06-08 18:13:04.413112951 +0000 UTC m=+18.267619745 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9986" (UniqueName: "kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986") pod "network-check-target-kznkg" (UID: "c85176c2-6792-4e89-83bb-4e2056fbe983") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:12:56.589457 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:56.589384 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:12:56.589597 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:56.589508 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kznkg" podUID="c85176c2-6792-4e89-83bb-4e2056fbe983" Jun 08 18:12:56.589908 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:56.589862 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:12:56.590043 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:56.589977 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjs4l" podUID="086390f2-8eb2-4903-a324-bf54451c435a" Jun 08 18:12:56.590106 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:56.590068 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:12:56.590175 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:56.590147 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s8pt8" podUID="eca9e438-ae5a-4645-ab87-b8d44d7d35af" Jun 08 18:12:58.587037 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:58.587004 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:12:58.587517 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:58.587004 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:12:58.587517 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:58.587130 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjs4l" podUID="086390f2-8eb2-4903-a324-bf54451c435a" Jun 08 18:12:58.587517 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:12:58.587004 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:12:58.587517 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:58.587197 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s8pt8" podUID="eca9e438-ae5a-4645-ab87-b8d44d7d35af" Jun 08 18:12:58.587517 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:12:58.587274 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kznkg" podUID="c85176c2-6792-4e89-83bb-4e2056fbe983" Jun 08 18:13:00.586905 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:00.586865 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:13:00.587401 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:00.586865 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:13:00.587401 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:00.587027 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjs4l" podUID="086390f2-8eb2-4903-a324-bf54451c435a" Jun 08 18:13:00.587401 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:00.587100 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kznkg" podUID="c85176c2-6792-4e89-83bb-4e2056fbe983" Jun 08 18:13:00.587401 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:00.586865 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:13:00.587401 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:00.587187 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s8pt8" podUID="eca9e438-ae5a-4645-ab87-b8d44d7d35af" Jun 08 18:13:02.586724 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:02.586685 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:13:02.586724 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:02.586712 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:13:02.587262 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:02.586688 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:13:02.587262 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:02.586847 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjs4l" podUID="086390f2-8eb2-4903-a324-bf54451c435a" Jun 08 18:13:02.587262 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:02.586927 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s8pt8" podUID="eca9e438-ae5a-4645-ab87-b8d44d7d35af" Jun 08 18:13:02.587262 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:02.586989 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kznkg" podUID="c85176c2-6792-4e89-83bb-4e2056fbe983" Jun 08 18:13:04.274275 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:04.274242 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs\") pod \"network-metrics-daemon-jjs4l\" (UID: \"086390f2-8eb2-4903-a324-bf54451c435a\") " pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:13:04.274720 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:04.274310 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret\") pod \"global-pull-secret-syncer-s8pt8\" (UID: \"eca9e438-ae5a-4645-ab87-b8d44d7d35af\") " pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:13:04.274720 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:04.274388 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:13:04.274720 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:04.274404 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 18:13:04.274720 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:04.274463 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs podName:086390f2-8eb2-4903-a324-bf54451c435a nodeName:}" failed. No retries permitted until 2026-06-08 18:13:20.274442311 +0000 UTC m=+34.128949087 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs") pod "network-metrics-daemon-jjs4l" (UID: "086390f2-8eb2-4903-a324-bf54451c435a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:13:04.274720 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:04.274486 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret podName:eca9e438-ae5a-4645-ab87-b8d44d7d35af nodeName:}" failed. No retries permitted until 2026-06-08 18:13:20.274476104 +0000 UTC m=+34.128982880 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret") pod "global-pull-secret-syncer-s8pt8" (UID: "eca9e438-ae5a-4645-ab87-b8d44d7d35af") : object "kube-system"/"original-pull-secret" not registered Jun 08 18:13:04.475855 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:04.475825 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9986\" (UniqueName: \"kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986\") pod \"network-check-target-kznkg\" (UID: \"c85176c2-6792-4e89-83bb-4e2056fbe983\") " pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:13:04.476046 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:04.476027 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 18:13:04.476091 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:04.476056 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 18:13:04.476091 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:04.476070 2572 projected.go:196] Error preparing data for projected volume kube-api-access-f9986 for pod openshift-network-diagnostics/network-check-target-kznkg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:13:04.476209 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:04.476133 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986 podName:c85176c2-6792-4e89-83bb-4e2056fbe983 nodeName:}" failed. No retries permitted until 2026-06-08 18:13:20.476114802 +0000 UTC m=+34.330621580 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9986" (UniqueName: "kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986") pod "network-check-target-kznkg" (UID: "c85176c2-6792-4e89-83bb-4e2056fbe983") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:13:04.587073 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:04.586997 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:13:04.587073 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:04.587005 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:13:04.587227 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:04.587005 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:13:04.587227 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:04.587135 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s8pt8" podUID="eca9e438-ae5a-4645-ab87-b8d44d7d35af" Jun 08 18:13:04.587298 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:04.587238 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjs4l" podUID="086390f2-8eb2-4903-a324-bf54451c435a" Jun 08 18:13:04.587332 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:04.587302 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kznkg" podUID="c85176c2-6792-4e89-83bb-4e2056fbe983" Jun 08 18:13:06.587890 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:06.587848 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:13:06.588453 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:06.588005 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:13:06.588453 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:06.588023 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:13:06.588453 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:06.588046 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjs4l" podUID="086390f2-8eb2-4903-a324-bf54451c435a" Jun 08 18:13:06.588453 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:06.588127 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kznkg" podUID="c85176c2-6792-4e89-83bb-4e2056fbe983" Jun 08 18:13:06.588453 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:06.588222 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s8pt8" podUID="eca9e438-ae5a-4645-ab87-b8d44d7d35af" Jun 08 18:13:08.586671 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.586507 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:13:08.587368 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.586550 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:13:08.587368 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:08.586765 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s8pt8" podUID="eca9e438-ae5a-4645-ab87-b8d44d7d35af" Jun 08 18:13:08.587368 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.586507 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:13:08.587368 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:08.586818 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kznkg" podUID="c85176c2-6792-4e89-83bb-4e2056fbe983" Jun 08 18:13:08.587368 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:08.586889 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjs4l" podUID="086390f2-8eb2-4903-a324-bf54451c435a" Jun 08 18:13:08.702830 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.702746 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qzwb8" event={"ID":"9e2844f5-ca13-4cce-8fab-61322b119c50","Type":"ContainerStarted","Data":"a42e1b636a997831b1e01de837d26efc577d0566e2c2f5d74a8d8619fa46b79e"} Jun 08 18:13:08.704313 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.704282 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p892x" event={"ID":"e27e2084-f792-4d85-9d63-99bb13520b73","Type":"ContainerStarted","Data":"b2edf2972821233d8d04bdbd36ab7c24d9fe84f3800d00729d049fd173496606"} Jun 08 18:13:08.705864 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.705837 2572 generic.go:356] "Generic (PLEG): container finished" podID="51fd7b9f-0366-434a-b567-33069764ed78" containerID="12c4aa965c6ad553b19bbc2d6377afbbaeed9ea8fe291d3965d6fa27b0c3d5a1" exitCode=0 Jun 08 18:13:08.705951 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.705916 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztjlr" event={"ID":"51fd7b9f-0366-434a-b567-33069764ed78","Type":"ContainerDied","Data":"12c4aa965c6ad553b19bbc2d6377afbbaeed9ea8fe291d3965d6fa27b0c3d5a1"} Jun 08 18:13:08.707719 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.707606 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z77dq" event={"ID":"1ad02886-8d99-4ad6-8103-a3eafea4af48","Type":"ContainerStarted","Data":"44b8522f7acead6b6737a5bd6fc2a9a897d3344a38445c24f091742169c5b340"} Jun 08 18:13:08.710703 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.710683 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 18:13:08.711109 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.711086 2572 generic.go:356] "Generic (PLEG): container finished" podID="ac833fb5-5b86-4ab4-aa93-2441a8a30ff1" containerID="8b03593a1b1bfd34ce6eaf7e996ca7ed2d209abc71d0a43aa454a42389d9655d" exitCode=1 Jun 08 18:13:08.711204 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.711153 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" event={"ID":"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1","Type":"ContainerStarted","Data":"9d0e6ccdf4bad79e6303e1ed6a3293d4abe1008a21f0a915df3a6622631354f0"} Jun 08 18:13:08.711267 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.711223 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" event={"ID":"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1","Type":"ContainerStarted","Data":"021f297762a068293ebde2754fafb59b131eadcdda3897d5300bc1ee0bf3463e"} Jun 08 18:13:08.711322 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.711284 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" event={"ID":"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1","Type":"ContainerStarted","Data":"34a9bd584c9630c6d901bfee64b00b6b463158550ae229942c4fcbb119239e1e"} Jun 08 18:13:08.711373 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.711332 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" event={"ID":"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1","Type":"ContainerStarted","Data":"7d090101667413b7c77ed7bd11f60f2f0bad3a3b920ab64027823d28ea6642e6"} Jun 08 18:13:08.711427 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.711378 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" event={"ID":"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1","Type":"ContainerDied","Data":"8b03593a1b1bfd34ce6eaf7e996ca7ed2d209abc71d0a43aa454a42389d9655d"} Jun 08 18:13:08.711427 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.711420 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" event={"ID":"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1","Type":"ContainerStarted","Data":"3b7f89cbd316ec837e0051cd79b174279e58326bef0228af701c8b0a25e105ab"} Jun 08 18:13:08.712756 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.712732 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bvg6f" event={"ID":"445f7d76-73b2-4d8c-aad9-9195be1bd5fa","Type":"ContainerStarted","Data":"b5160cd19d516e46fdb7a097ed741a66c4c193eb218dbe4df5bb436cf3e463df"} Jun 08 18:13:08.714499 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.714478 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" event={"ID":"78bb4fa4-4698-4288-a89c-fad4b5b5a825","Type":"ContainerStarted","Data":"6a8affe86492372cccb59a47e0b9281bea12ef5a78fcade32b83d7b5883b90c4"} Jun 08 18:13:08.715859 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.715842 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" event={"ID":"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1","Type":"ContainerStarted","Data":"ff136a2a8d196557d0b3285128f1ea0992b443eacce89123aa7ca0eb26f38272"} Jun 08 18:13:08.722988 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.722924 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qzwb8" podStartSLOduration=4.293826555 podStartE2EDuration="22.722912455s" totalImagesPullingTime="18.4290859s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:13:08.722359963 +0000 UTC m=+22.576866773" watchObservedRunningTime="2026-06-08 18:13:08.722912455 +0000 UTC m=+22.577419254" Jun 08 18:13:08.740247 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.740203 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4l6bl" podStartSLOduration=4.324547566 podStartE2EDuration="22.740188191s" totalImagesPullingTime="18.415640625s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:13:08.739856698 +0000 UTC m=+22.594363497" watchObservedRunningTime="2026-06-08 18:13:08.740188191 +0000 UTC m=+22.594694991" Jun 08 18:13:08.773170 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.773126 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-z77dq" podStartSLOduration=4.375509809 podStartE2EDuration="22.773113538s" totalImagesPullingTime="18.397603729s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:13:08.772323158 +0000 UTC m=+22.626829956" watchObservedRunningTime="2026-06-08 18:13:08.773113538 +0000 UTC m=+22.627620336" Jun 08 18:13:08.788996 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.788941 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-p892x" podStartSLOduration=12.777212212 podStartE2EDuration="22.788929421s" totalImagesPullingTime="10.011717209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:13:08.788332273 +0000 UTC m=+22.642839070" watchObservedRunningTime="2026-06-08 18:13:08.788929421 +0000 UTC m=+22.643436251" Jun 08 18:13:08.807669 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:08.807637 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bvg6f" podStartSLOduration=4.398410716 podStartE2EDuration="22.807626956s" totalImagesPullingTime="18.40921624s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:13:08.807596406 +0000 UTC m=+22.662103215" watchObservedRunningTime="2026-06-08 18:13:08.807626956 +0000 UTC m=+22.662133755" Jun 08 18:13:09.538073 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:09.537912 2572 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 08 18:13:09.589447 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:09.589335 2572 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-08T18:13:09.538068175Z","UUID":"eb77d55d-881e-46f3-8a93-cc0bfc0b7f22","Handler":null,"Name":"","Endpoint":""} Jun 08 18:13:09.590612 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:09.590595 2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 08 18:13:09.590695 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:09.590621 2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 08 18:13:09.719007 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:09.718916 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pvn2c" event={"ID":"0f417f4b-e7ad-4076-bd6f-039a41d12fe9","Type":"ContainerStarted","Data":"e6a127b41df4c295974ff28002c9808ccd6602aec54f1f786d35b35d5cbea647"} Jun 08 18:13:09.720597 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:09.720568 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" event={"ID":"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1","Type":"ContainerStarted","Data":"d07f17e62f879d3a7fba6e676e6fc8c1477a06eae4696ba616c984d27753a3a3"} Jun 08 18:13:09.734004 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:09.733934 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pvn2c" podStartSLOduration=5.334501255 podStartE2EDuration="23.733917078s" totalImagesPullingTime="18.399415823s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:13:09.733736051 +0000 UTC m=+23.588242848" watchObservedRunningTime="2026-06-08 18:13:09.733917078 +0000 UTC m=+23.588423879" Jun 08 18:13:10.586392 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:10.586343 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:13:10.586392 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:10.586387 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:13:10.586637 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:10.586343 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:13:10.586637 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:10.586481 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kznkg" podUID="c85176c2-6792-4e89-83bb-4e2056fbe983" Jun 08 18:13:10.586637 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:10.586515 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s8pt8" podUID="eca9e438-ae5a-4645-ab87-b8d44d7d35af" Jun 08 18:13:10.586637 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:10.586587 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjs4l" podUID="086390f2-8eb2-4903-a324-bf54451c435a" Jun 08 18:13:11.728815 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:11.728783 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 18:13:11.729470 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:11.729273 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" event={"ID":"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1","Type":"ContainerStarted","Data":"44a2a6cb21dec6d20c1be499113e2d63c4b5c037b6caeb50d085c9937e4b8033"} Jun 08 18:13:11.731344 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:11.731314 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" event={"ID":"367102c3-3e6a-4c17-9d5d-a8e6a2df7fb1","Type":"ContainerStarted","Data":"e6e96a4edbe4db2abd1af4cbec168d967b55505577762dd7bbd0f0993b1384bb"} Jun 08 18:13:11.752874 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:11.752819 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r8nlc" podStartSLOduration=4.949106632 podStartE2EDuration="25.752797904s" totalImagesPullingTime="20.803691272s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:12:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:13:11.752472358 +0000 UTC m=+25.606979156" watchObservedRunningTime="2026-06-08 18:13:11.752797904 +0000 UTC m=+25.607304703" Jun 08 18:13:11.793759 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:11.793725 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-p892x" Jun 08 18:13:11.794461 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:11.794438 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-p892x" Jun 08 18:13:12.586829 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:12.586794 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:13:12.587030 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:12.586840 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:13:12.587030 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:12.586790 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:13:12.587030 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:12.586909 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s8pt8" podUID="eca9e438-ae5a-4645-ab87-b8d44d7d35af" Jun 08 18:13:12.587030 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:12.586975 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kznkg" podUID="c85176c2-6792-4e89-83bb-4e2056fbe983" Jun 08 18:13:12.587252 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:12.587042 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjs4l" podUID="086390f2-8eb2-4903-a324-bf54451c435a" Jun 08 18:13:12.734029 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:12.733997 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-p892x" Jun 08 18:13:12.734582 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:12.734324 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-p892x" Jun 08 18:13:13.739084 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:13.738884 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 18:13:13.739535 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:13.739440 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" event={"ID":"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1","Type":"ContainerStarted","Data":"a3d8974b8298d33627d5b1fb14fa01e3713c487c5138b10410655510cf0c91d3"} Jun 08 18:13:13.739682 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:13.739660 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:13:13.739840 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:13.739721 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:13:13.739840 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:13.739822 2572 scope.go:117] "RemoveContainer" containerID="8b03593a1b1bfd34ce6eaf7e996ca7ed2d209abc71d0a43aa454a42389d9655d" Jun 08 18:13:13.741483 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:13.741459 2572 generic.go:356] "Generic (PLEG): container finished" podID="51fd7b9f-0366-434a-b567-33069764ed78" containerID="af5f6b042c05cbdaf3f56c6473d1d4dac35861122015581f6cd51c38354bb19f" exitCode=0 Jun 08 18:13:13.741569 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:13.741550 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztjlr" event={"ID":"51fd7b9f-0366-434a-b567-33069764ed78","Type":"ContainerDied","Data":"af5f6b042c05cbdaf3f56c6473d1d4dac35861122015581f6cd51c38354bb19f"} Jun 08 18:13:13.755854 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:13.755832 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:13:14.589460 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:14.589421 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:13:14.589601 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:14.589563 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjs4l" podUID="086390f2-8eb2-4903-a324-bf54451c435a" Jun 08 18:13:14.589667 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:14.589642 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:13:14.589733 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:14.589710 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s8pt8" podUID="eca9e438-ae5a-4645-ab87-b8d44d7d35af" Jun 08 18:13:14.589785 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:14.589768 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:13:14.589848 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:14.589826 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kznkg" podUID="c85176c2-6792-4e89-83bb-4e2056fbe983" Jun 08 18:13:14.746409 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:14.746327 2572 generic.go:356] "Generic (PLEG): container finished" podID="51fd7b9f-0366-434a-b567-33069764ed78" containerID="0d11d8a38a113f8422e5e22d9fbb44f9da866f68ef31b64a10331006603ef2e6" exitCode=0 Jun 08 18:13:14.746804 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:14.746415 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztjlr" event={"ID":"51fd7b9f-0366-434a-b567-33069764ed78","Type":"ContainerDied","Data":"0d11d8a38a113f8422e5e22d9fbb44f9da866f68ef31b64a10331006603ef2e6"} Jun 08 18:13:14.750537 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:14.750521 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 18:13:14.751003 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:14.750979 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" event={"ID":"ac833fb5-5b86-4ab4-aa93-2441a8a30ff1","Type":"ContainerStarted","Data":"49cc28b4b8dbc28ce41b839bc2e95569697b022de494e9b74e5e009ff3bbd1e6"} Jun 08 18:13:14.751281 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:14.751257 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:13:14.766177 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:14.766029 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:13:14.792170 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:14.792122 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" podStartSLOduration=10.313565269 podStartE2EDuration="28.792107032s" totalImagesPullingTime="18.478541763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:13:14.790578544 +0000 UTC m=+28.645085342" watchObservedRunningTime="2026-06-08 18:13:14.792107032 +0000 UTC m=+28.646613849" Jun 08 18:13:15.224183 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:15.224152 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-s8pt8"] Jun 08 18:13:15.224333 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:15.224269 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:13:15.224416 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:15.224389 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s8pt8" podUID="eca9e438-ae5a-4645-ab87-b8d44d7d35af" Jun 08 18:13:15.227419 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:15.227223 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kznkg"] Jun 08 18:13:15.227419 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:15.227321 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:13:15.227598 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:15.227471 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kznkg" podUID="c85176c2-6792-4e89-83bb-4e2056fbe983" Jun 08 18:13:15.228069 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:15.228034 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jjs4l"] Jun 08 18:13:15.228188 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:15.228128 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:13:15.228259 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:15.228234 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjs4l" podUID="086390f2-8eb2-4903-a324-bf54451c435a" Jun 08 18:13:15.755376 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:15.755286 2572 generic.go:356] "Generic (PLEG): container finished" podID="51fd7b9f-0366-434a-b567-33069764ed78" containerID="e70d9639580611cfba00ea44937e4a5c2d33a9d59a8c76134d4d249d717e00c0" exitCode=0 Jun 08 18:13:15.755858 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:15.755374 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztjlr" event={"ID":"51fd7b9f-0366-434a-b567-33069764ed78","Type":"ContainerDied","Data":"e70d9639580611cfba00ea44937e4a5c2d33a9d59a8c76134d4d249d717e00c0"} Jun 08 18:13:16.587657 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:16.587274 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:13:16.587657 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:16.587378 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s8pt8" podUID="eca9e438-ae5a-4645-ab87-b8d44d7d35af" Jun 08 18:13:16.587657 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:16.587481 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:13:16.587657 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:16.587575 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kznkg" podUID="c85176c2-6792-4e89-83bb-4e2056fbe983" Jun 08 18:13:17.587201 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:17.587117 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:13:17.587837 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:17.587254 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjs4l" podUID="086390f2-8eb2-4903-a324-bf54451c435a" Jun 08 18:13:18.587271 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:18.587236 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:13:18.587727 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:18.587370 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s8pt8" podUID="eca9e438-ae5a-4645-ab87-b8d44d7d35af" Jun 08 18:13:18.587727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:18.587428 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:13:18.587727 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:18.587571 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kznkg" podUID="c85176c2-6792-4e89-83bb-4e2056fbe983" Jun 08 18:13:19.586287 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:19.586211 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:13:19.586480 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:19.586330 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jjs4l" podUID="086390f2-8eb2-4903-a324-bf54451c435a" Jun 08 18:13:20.291354 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:20.291146 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret\") pod \"global-pull-secret-syncer-s8pt8\" (UID: \"eca9e438-ae5a-4645-ab87-b8d44d7d35af\") " pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:13:20.291792 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:20.291383 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs\") pod \"network-metrics-daemon-jjs4l\" (UID: \"086390f2-8eb2-4903-a324-bf54451c435a\") " pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:13:20.291792 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:20.291308 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 18:13:20.291792 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:20.291496 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret podName:eca9e438-ae5a-4645-ab87-b8d44d7d35af nodeName:}" failed. No retries permitted until 2026-06-08 18:13:52.291480029 +0000 UTC m=+66.145986808 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret") pod "global-pull-secret-syncer-s8pt8" (UID: "eca9e438-ae5a-4645-ab87-b8d44d7d35af") : object "kube-system"/"original-pull-secret" not registered Jun 08 18:13:20.291792 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:20.291512 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:13:20.291792 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:20.291562 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs podName:086390f2-8eb2-4903-a324-bf54451c435a nodeName:}" failed. No retries permitted until 2026-06-08 18:13:52.291546918 +0000 UTC m=+66.146053711 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs") pod "network-metrics-daemon-jjs4l" (UID: "086390f2-8eb2-4903-a324-bf54451c435a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 18:13:20.493800 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:20.493760 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9986\" (UniqueName: \"kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986\") pod \"network-check-target-kznkg\" (UID: \"c85176c2-6792-4e89-83bb-4e2056fbe983\") " pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:13:20.493947 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:20.493906 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 18:13:20.493947 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:20.493932 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 18:13:20.493947 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:20.493945 2572 projected.go:196] Error preparing data for projected volume kube-api-access-f9986 for pod openshift-network-diagnostics/network-check-target-kznkg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:13:20.494116 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:20.494020 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986 podName:c85176c2-6792-4e89-83bb-4e2056fbe983 nodeName:}" failed. No retries permitted until 2026-06-08 18:13:52.494004323 +0000 UTC m=+66.348511099 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9986" (UniqueName: "kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986") pod "network-check-target-kznkg" (UID: "c85176c2-6792-4e89-83bb-4e2056fbe983") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 18:13:20.590257 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:20.590146 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:13:20.590412 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:20.590265 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kznkg" podUID="c85176c2-6792-4e89-83bb-4e2056fbe983" Jun 08 18:13:20.590412 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:20.590328 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:13:20.590502 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:20.590432 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s8pt8" podUID="eca9e438-ae5a-4645-ab87-b8d44d7d35af" Jun 08 18:13:20.943131 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:20.943057 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-194.ec2.internal" event="NodeReady" Jun 08 18:13:20.943272 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:20.943217 2572 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 08 18:13:20.991671 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:20.991642 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-n4m82"] Jun 08 18:13:20.993880 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:20.993859 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n4m82" Jun 08 18:13:20.995969 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:20.995937 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 08 18:13:20.996886 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:20.996865 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bl784"] Jun 08 18:13:20.997139 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:20.997122 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 08 18:13:20.997193 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:20.997141 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s24pr\"" Jun 08 18:13:20.998465 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:20.998450 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bl784" Jun 08 18:13:21.000175 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.000158 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 08 18:13:21.000254 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.000223 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 08 18:13:21.000374 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.000360 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-blwgz\"" Jun 08 18:13:21.000453 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.000437 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 08 18:13:21.008037 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.008018 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n4m82"] Jun 08 18:13:21.012177 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.012153 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bl784"] Jun 08 18:13:21.097270 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.097240 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb9lr\" (UniqueName: \"kubernetes.io/projected/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-kube-api-access-kb9lr\") pod \"dns-default-n4m82\" (UID: \"c2665ffe-cccc-432a-85a7-a5244f0b2bc8\") " pod="openshift-dns/dns-default-n4m82" Jun 08 18:13:21.097394 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.097299 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a6fe03e-3966-431d-99ff-43a27c4d0249-cert\") pod \"ingress-canary-bl784\" (UID: \"0a6fe03e-3966-431d-99ff-43a27c4d0249\") " pod="openshift-ingress-canary/ingress-canary-bl784" Jun 08 18:13:21.097394 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.097338 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-config-volume\") pod \"dns-default-n4m82\" (UID: \"c2665ffe-cccc-432a-85a7-a5244f0b2bc8\") " pod="openshift-dns/dns-default-n4m82" Jun 08 18:13:21.097394 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.097373 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-tmp-dir\") pod \"dns-default-n4m82\" (UID: \"c2665ffe-cccc-432a-85a7-a5244f0b2bc8\") " pod="openshift-dns/dns-default-n4m82" Jun 08 18:13:21.097394 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.097391 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls\") pod \"dns-default-n4m82\" (UID: \"c2665ffe-cccc-432a-85a7-a5244f0b2bc8\") " pod="openshift-dns/dns-default-n4m82" Jun 08 18:13:21.097566 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.097494 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7xs5\" (UniqueName: \"kubernetes.io/projected/0a6fe03e-3966-431d-99ff-43a27c4d0249-kube-api-access-w7xs5\") pod \"ingress-canary-bl784\" (UID: \"0a6fe03e-3966-431d-99ff-43a27c4d0249\") " pod="openshift-ingress-canary/ingress-canary-bl784" Jun 08 18:13:21.198195 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.198106 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-tmp-dir\") pod \"dns-default-n4m82\" (UID: \"c2665ffe-cccc-432a-85a7-a5244f0b2bc8\") " pod="openshift-dns/dns-default-n4m82" Jun 08 18:13:21.198195 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.198152 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls\") pod \"dns-default-n4m82\" (UID: \"c2665ffe-cccc-432a-85a7-a5244f0b2bc8\") " pod="openshift-dns/dns-default-n4m82" Jun 08 18:13:21.198381 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.198200 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7xs5\" (UniqueName: \"kubernetes.io/projected/0a6fe03e-3966-431d-99ff-43a27c4d0249-kube-api-access-w7xs5\") pod \"ingress-canary-bl784\" (UID: \"0a6fe03e-3966-431d-99ff-43a27c4d0249\") " pod="openshift-ingress-canary/ingress-canary-bl784" Jun 08 18:13:21.198381 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.198229 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb9lr\" (UniqueName: \"kubernetes.io/projected/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-kube-api-access-kb9lr\") pod \"dns-default-n4m82\" (UID: \"c2665ffe-cccc-432a-85a7-a5244f0b2bc8\") " pod="openshift-dns/dns-default-n4m82" Jun 08 18:13:21.198381 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:21.198354 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 18:13:21.198506 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.198374 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a6fe03e-3966-431d-99ff-43a27c4d0249-cert\") pod \"ingress-canary-bl784\" (UID: \"0a6fe03e-3966-431d-99ff-43a27c4d0249\") " pod="openshift-ingress-canary/ingress-canary-bl784" Jun 08 18:13:21.198506 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.198441 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-config-volume\") pod \"dns-default-n4m82\" (UID: \"c2665ffe-cccc-432a-85a7-a5244f0b2bc8\") " pod="openshift-dns/dns-default-n4m82" Jun 08 18:13:21.198506 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:21.198464 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls podName:c2665ffe-cccc-432a-85a7-a5244f0b2bc8 nodeName:}" failed. No retries permitted until 2026-06-08 18:13:21.698441508 +0000 UTC m=+35.552948305 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls") pod "dns-default-n4m82" (UID: "c2665ffe-cccc-432a-85a7-a5244f0b2bc8") : secret "dns-default-metrics-tls" not found Jun 08 18:13:21.198506 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.198479 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-tmp-dir\") pod \"dns-default-n4m82\" (UID: \"c2665ffe-cccc-432a-85a7-a5244f0b2bc8\") " pod="openshift-dns/dns-default-n4m82" Jun 08 18:13:21.198976 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.198941 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-config-volume\") pod \"dns-default-n4m82\" (UID: \"c2665ffe-cccc-432a-85a7-a5244f0b2bc8\") " pod="openshift-dns/dns-default-n4m82" Jun 08 18:13:21.202628 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.202609 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a6fe03e-3966-431d-99ff-43a27c4d0249-cert\") pod \"ingress-canary-bl784\" (UID: \"0a6fe03e-3966-431d-99ff-43a27c4d0249\") " pod="openshift-ingress-canary/ingress-canary-bl784" Jun 08 18:13:21.206494 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.206470 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb9lr\" (UniqueName: \"kubernetes.io/projected/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-kube-api-access-kb9lr\") pod \"dns-default-n4m82\" (UID: \"c2665ffe-cccc-432a-85a7-a5244f0b2bc8\") " pod="openshift-dns/dns-default-n4m82" Jun 08 18:13:21.206634 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.206614 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7xs5\" (UniqueName: \"kubernetes.io/projected/0a6fe03e-3966-431d-99ff-43a27c4d0249-kube-api-access-w7xs5\") pod \"ingress-canary-bl784\" (UID: \"0a6fe03e-3966-431d-99ff-43a27c4d0249\") " pod="openshift-ingress-canary/ingress-canary-bl784" Jun 08 18:13:21.338611 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.338572 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bl784" Jun 08 18:13:21.473112 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.473032 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bl784"] Jun 08 18:13:21.477655 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:13:21.477631 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a6fe03e_3966_431d_99ff_43a27c4d0249.slice/crio-fd009f13ee0f467b2b4d095da784ae87d1ba499dca32054ca5034eee540bab97 WatchSource:0}: Error finding container fd009f13ee0f467b2b4d095da784ae87d1ba499dca32054ca5034eee540bab97: Status 404 returned error can't find the container with id fd009f13ee0f467b2b4d095da784ae87d1ba499dca32054ca5034eee540bab97 Jun 08 18:13:21.586880 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.586848 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:13:21.589164 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.589140 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-58l4q\"" Jun 08 18:13:21.589268 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.589172 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 08 18:13:21.701826 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.701788 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls\") pod \"dns-default-n4m82\" (UID: \"c2665ffe-cccc-432a-85a7-a5244f0b2bc8\") " pod="openshift-dns/dns-default-n4m82" Jun 08 18:13:21.702024 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:21.701903 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 18:13:21.702024 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:21.701984 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls podName:c2665ffe-cccc-432a-85a7-a5244f0b2bc8 nodeName:}" failed. No retries permitted until 2026-06-08 18:13:22.701954009 +0000 UTC m=+36.556460798 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls") pod "dns-default-n4m82" (UID: "c2665ffe-cccc-432a-85a7-a5244f0b2bc8") : secret "dns-default-metrics-tls" not found Jun 08 18:13:21.772698 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.772613 2572 generic.go:356] "Generic (PLEG): container finished" podID="51fd7b9f-0366-434a-b567-33069764ed78" containerID="93090059f85754f296ad6251cbed574e2ce7236926b8898c040d42884cec8647" exitCode=0 Jun 08 18:13:21.772837 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.772699 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztjlr" event={"ID":"51fd7b9f-0366-434a-b567-33069764ed78","Type":"ContainerDied","Data":"93090059f85754f296ad6251cbed574e2ce7236926b8898c040d42884cec8647"} Jun 08 18:13:21.773898 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:21.773865 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bl784" event={"ID":"0a6fe03e-3966-431d-99ff-43a27c4d0249","Type":"ContainerStarted","Data":"fd009f13ee0f467b2b4d095da784ae87d1ba499dca32054ca5034eee540bab97"} Jun 08 18:13:22.051558 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.051255 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65584f7cd6-whmlg"] Jun 08 18:13:22.053427 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.053405 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65584f7cd6-whmlg" Jun 08 18:13:22.055865 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.055836 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 08 18:13:22.056169 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.056152 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-nlxh5\"" Jun 08 18:13:22.056255 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.056215 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 08 18:13:22.056317 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.056284 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jun 08 18:13:22.056511 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.056497 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 08 18:13:22.066774 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.066749 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65584f7cd6-whmlg"] Jun 08 18:13:22.205022 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.204983 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7fcdb8ae-ba39-465c-9112-3f39e2e9032a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-65584f7cd6-whmlg\" (UID: \"7fcdb8ae-ba39-465c-9112-3f39e2e9032a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65584f7cd6-whmlg" Jun 08 18:13:22.205198 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.205127 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgqbm\" (UniqueName: \"kubernetes.io/projected/7fcdb8ae-ba39-465c-9112-3f39e2e9032a-kube-api-access-zgqbm\") pod \"managed-serviceaccount-addon-agent-65584f7cd6-whmlg\" (UID: \"7fcdb8ae-ba39-465c-9112-3f39e2e9032a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65584f7cd6-whmlg" Jun 08 18:13:22.306500 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.306417 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgqbm\" (UniqueName: \"kubernetes.io/projected/7fcdb8ae-ba39-465c-9112-3f39e2e9032a-kube-api-access-zgqbm\") pod \"managed-serviceaccount-addon-agent-65584f7cd6-whmlg\" (UID: \"7fcdb8ae-ba39-465c-9112-3f39e2e9032a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65584f7cd6-whmlg" Jun 08 18:13:22.306500 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.306480 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7fcdb8ae-ba39-465c-9112-3f39e2e9032a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-65584f7cd6-whmlg\" (UID: \"7fcdb8ae-ba39-465c-9112-3f39e2e9032a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65584f7cd6-whmlg" Jun 08 18:13:22.310400 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.310369 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7fcdb8ae-ba39-465c-9112-3f39e2e9032a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-65584f7cd6-whmlg\" (UID: \"7fcdb8ae-ba39-465c-9112-3f39e2e9032a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65584f7cd6-whmlg" Jun 08 18:13:22.315303 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.315275 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgqbm\" (UniqueName: \"kubernetes.io/projected/7fcdb8ae-ba39-465c-9112-3f39e2e9032a-kube-api-access-zgqbm\") pod \"managed-serviceaccount-addon-agent-65584f7cd6-whmlg\" (UID: \"7fcdb8ae-ba39-465c-9112-3f39e2e9032a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65584f7cd6-whmlg" Jun 08 18:13:22.376193 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.376159 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65584f7cd6-whmlg" Jun 08 18:13:22.499304 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.499269 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65584f7cd6-whmlg"] Jun 08 18:13:22.505145 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:13:22.505102 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7fcdb8ae_ba39_465c_9112_3f39e2e9032a.slice/crio-1ec75d2f7bd94ad6a89d2d57bbf857c42a5745e04a55e6c2445d8a31537f85b4 WatchSource:0}: Error finding container 1ec75d2f7bd94ad6a89d2d57bbf857c42a5745e04a55e6c2445d8a31537f85b4: Status 404 returned error can't find the container with id 1ec75d2f7bd94ad6a89d2d57bbf857c42a5745e04a55e6c2445d8a31537f85b4 Jun 08 18:13:22.587095 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.587012 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:13:22.587237 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.587012 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:13:22.589595 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.589572 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 08 18:13:22.589731 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.589606 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n4vlz\"" Jun 08 18:13:22.589731 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.589694 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 08 18:13:22.589839 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.589694 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 08 18:13:22.708862 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.708818 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls\") pod \"dns-default-n4m82\" (UID: \"c2665ffe-cccc-432a-85a7-a5244f0b2bc8\") " pod="openshift-dns/dns-default-n4m82" Jun 08 18:13:22.709059 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:22.708984 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 18:13:22.709059 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:22.709050 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls podName:c2665ffe-cccc-432a-85a7-a5244f0b2bc8 nodeName:}" failed. No retries permitted until 2026-06-08 18:13:24.709034146 +0000 UTC m=+38.563540921 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls") pod "dns-default-n4m82" (UID: "c2665ffe-cccc-432a-85a7-a5244f0b2bc8") : secret "dns-default-metrics-tls" not found Jun 08 18:13:22.780017 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.779976 2572 generic.go:356] "Generic (PLEG): container finished" podID="51fd7b9f-0366-434a-b567-33069764ed78" containerID="6f54d1bb3461de2dba36d90384080f6c9360321e1c56731c433d9df90796316d" exitCode=0 Jun 08 18:13:22.780017 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.779999 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztjlr" event={"ID":"51fd7b9f-0366-434a-b567-33069764ed78","Type":"ContainerDied","Data":"6f54d1bb3461de2dba36d90384080f6c9360321e1c56731c433d9df90796316d"} Jun 08 18:13:22.781338 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:22.781313 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65584f7cd6-whmlg" event={"ID":"7fcdb8ae-ba39-465c-9112-3f39e2e9032a","Type":"ContainerStarted","Data":"1ec75d2f7bd94ad6a89d2d57bbf857c42a5745e04a55e6c2445d8a31537f85b4"} Jun 08 18:13:23.787173 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:23.787142 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztjlr" event={"ID":"51fd7b9f-0366-434a-b567-33069764ed78","Type":"ContainerStarted","Data":"4e11f9039ea6941c690ef2da5a2f25155aa119cad45417111134ad73b1618ab0"} Jun 08 18:13:23.810531 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:23.810440 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ztjlr" podStartSLOduration=8.396738862 podStartE2EDuration="37.810419114s" totalImagesPullingTime="29.413680252s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:12:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:13:23.809378232 +0000 UTC m=+37.663885043" watchObservedRunningTime="2026-06-08 18:13:23.810419114 +0000 UTC m=+37.664925914" Jun 08 18:13:24.726374 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:24.726334 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls\") pod \"dns-default-n4m82\" (UID: \"c2665ffe-cccc-432a-85a7-a5244f0b2bc8\") " pod="openshift-dns/dns-default-n4m82" Jun 08 18:13:24.726548 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:24.726511 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 18:13:24.726612 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:24.726592 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls podName:c2665ffe-cccc-432a-85a7-a5244f0b2bc8 nodeName:}" failed. No retries permitted until 2026-06-08 18:13:28.726570197 +0000 UTC m=+42.581076974 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls") pod "dns-default-n4m82" (UID: "c2665ffe-cccc-432a-85a7-a5244f0b2bc8") : secret "dns-default-metrics-tls" not found Jun 08 18:13:25.796510 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:25.796207 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bl784" event={"ID":"0a6fe03e-3966-431d-99ff-43a27c4d0249","Type":"ContainerStarted","Data":"f38217c1e31f4edc1bdcd1f81da6129410770665b20076c561acfbf7c0f61aa7"} Jun 08 18:13:25.797719 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:25.797695 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65584f7cd6-whmlg" event={"ID":"7fcdb8ae-ba39-465c-9112-3f39e2e9032a","Type":"ContainerStarted","Data":"19c7d7dfda0ae963924f283752a1de51a965c9284302eca298dad99fd101dc9a"} Jun 08 18:13:25.813045 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:25.813000 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bl784" podStartSLOduration=2.24253116 podStartE2EDuration="5.812982324s" totalImagesPullingTime="3.570451164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:13:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:13:25.812172487 +0000 UTC m=+39.666679284" watchObservedRunningTime="2026-06-08 18:13:25.812982324 +0000 UTC m=+39.667489120" Jun 08 18:13:25.828840 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:25.828788 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65584f7cd6-whmlg" podStartSLOduration=0.885514718 podStartE2EDuration="3.828768446s" totalImagesPullingTime="2.943253728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:13:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:13:25.828152215 +0000 UTC m=+39.682659051" watchObservedRunningTime="2026-06-08 18:13:25.828768446 +0000 UTC m=+39.683275243" Jun 08 18:13:28.758091 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:28.758051 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls\") pod \"dns-default-n4m82\" (UID: \"c2665ffe-cccc-432a-85a7-a5244f0b2bc8\") " pod="openshift-dns/dns-default-n4m82" Jun 08 18:13:28.758461 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:28.758188 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 18:13:28.758461 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:28.758245 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls podName:c2665ffe-cccc-432a-85a7-a5244f0b2bc8 nodeName:}" failed. No retries permitted until 2026-06-08 18:13:36.758229011 +0000 UTC m=+50.612735788 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls") pod "dns-default-n4m82" (UID: "c2665ffe-cccc-432a-85a7-a5244f0b2bc8") : secret "dns-default-metrics-tls" not found Jun 08 18:13:36.816341 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:36.816305 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls\") pod \"dns-default-n4m82\" (UID: \"c2665ffe-cccc-432a-85a7-a5244f0b2bc8\") " pod="openshift-dns/dns-default-n4m82" Jun 08 18:13:36.816876 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:36.816416 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 18:13:36.816876 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:36.816469 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls podName:c2665ffe-cccc-432a-85a7-a5244f0b2bc8 nodeName:}" failed. No retries permitted until 2026-06-08 18:13:52.816454495 +0000 UTC m=+66.670961279 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls") pod "dns-default-n4m82" (UID: "c2665ffe-cccc-432a-85a7-a5244f0b2bc8") : secret "dns-default-metrics-tls" not found Jun 08 18:13:46.767706 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:46.767677 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jpwxq" Jun 08 18:13:52.330510 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:52.330467 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs\") pod \"network-metrics-daemon-jjs4l\" (UID: \"086390f2-8eb2-4903-a324-bf54451c435a\") " pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:13:52.331109 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:52.330569 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret\") pod \"global-pull-secret-syncer-s8pt8\" (UID: \"eca9e438-ae5a-4645-ab87-b8d44d7d35af\") " pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:13:52.332743 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:52.332724 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 08 18:13:52.332810 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:52.332769 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 08 18:13:52.341485 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:52.341460 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 08 18:13:52.341585 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:52.341549 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs podName:086390f2-8eb2-4903-a324-bf54451c435a nodeName:}" failed. No retries permitted until 2026-06-08 18:14:56.341525502 +0000 UTC m=+130.196032294 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs") pod "network-metrics-daemon-jjs4l" (UID: "086390f2-8eb2-4903-a324-bf54451c435a") : secret "metrics-daemon-secret" not found Jun 08 18:13:52.344433 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:52.344411 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eca9e438-ae5a-4645-ab87-b8d44d7d35af-original-pull-secret\") pod \"global-pull-secret-syncer-s8pt8\" (UID: \"eca9e438-ae5a-4645-ab87-b8d44d7d35af\") " pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:13:52.532578 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:52.532529 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9986\" (UniqueName: \"kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986\") pod \"network-check-target-kznkg\" (UID: \"c85176c2-6792-4e89-83bb-4e2056fbe983\") " pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:13:52.535094 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:52.535065 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 08 18:13:52.544816 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:52.544790 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 08 18:13:52.555715 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:52.555687 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9986\" (UniqueName: \"kubernetes.io/projected/c85176c2-6792-4e89-83bb-4e2056fbe983-kube-api-access-f9986\") pod \"network-check-target-kznkg\" (UID: \"c85176c2-6792-4e89-83bb-4e2056fbe983\") " pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:13:52.598538 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:52.598462 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s8pt8" Jun 08 18:13:52.609823 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:52.609800 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n4vlz\"" Jun 08 18:13:52.617706 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:52.617684 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:13:52.748055 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:52.748022 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-s8pt8"] Jun 08 18:13:52.752198 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:13:52.752170 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeca9e438_ae5a_4645_ab87_b8d44d7d35af.slice/crio-a5a9561d09e276825de20b06ae600803f34fd74ca4610ffd0fb514774b093349 WatchSource:0}: Error finding container a5a9561d09e276825de20b06ae600803f34fd74ca4610ffd0fb514774b093349: Status 404 returned error can't find the container with id a5a9561d09e276825de20b06ae600803f34fd74ca4610ffd0fb514774b093349 Jun 08 18:13:52.765866 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:52.765841 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kznkg"] Jun 08 18:13:52.769444 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:13:52.769419 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc85176c2_6792_4e89_83bb_4e2056fbe983.slice/crio-be04b3b6be9ead206ec1f72b3c165111b8494eaae4bd8fbfe206f311904ec9a2 WatchSource:0}: Error finding container be04b3b6be9ead206ec1f72b3c165111b8494eaae4bd8fbfe206f311904ec9a2: Status 404 returned error can't find the container with id be04b3b6be9ead206ec1f72b3c165111b8494eaae4bd8fbfe206f311904ec9a2 Jun 08 18:13:52.835205 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:52.835165 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls\") pod \"dns-default-n4m82\" (UID: \"c2665ffe-cccc-432a-85a7-a5244f0b2bc8\") " pod="openshift-dns/dns-default-n4m82" Jun 08 18:13:52.835374 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:52.835288 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 18:13:52.835374 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:13:52.835357 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls podName:c2665ffe-cccc-432a-85a7-a5244f0b2bc8 nodeName:}" failed. No retries permitted until 2026-06-08 18:14:24.835336715 +0000 UTC m=+98.689843491 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls") pod "dns-default-n4m82" (UID: "c2665ffe-cccc-432a-85a7-a5244f0b2bc8") : secret "dns-default-metrics-tls" not found Jun 08 18:13:52.862224 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:52.862141 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-s8pt8" event={"ID":"eca9e438-ae5a-4645-ab87-b8d44d7d35af","Type":"ContainerStarted","Data":"a5a9561d09e276825de20b06ae600803f34fd74ca4610ffd0fb514774b093349"} Jun 08 18:13:52.863274 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:52.863251 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kznkg" event={"ID":"c85176c2-6792-4e89-83bb-4e2056fbe983","Type":"ContainerStarted","Data":"be04b3b6be9ead206ec1f72b3c165111b8494eaae4bd8fbfe206f311904ec9a2"} Jun 08 18:13:57.879932 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:57.879888 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-s8pt8" event={"ID":"eca9e438-ae5a-4645-ab87-b8d44d7d35af","Type":"ContainerStarted","Data":"03c4eaf6e25aef891676c2795434af3840bb4ef485777815a2935808aedc45fb"} Jun 08 18:13:57.881290 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:57.881264 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kznkg" event={"ID":"c85176c2-6792-4e89-83bb-4e2056fbe983","Type":"ContainerStarted","Data":"5c9a30dc9c943f4211efb340b8b25a3f32496a2f439403121138358b3a2160a1"} Jun 08 18:13:57.881422 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:57.881400 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:13:57.898681 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:57.898636 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-s8pt8" podStartSLOduration=67.411885835 podStartE2EDuration="1m11.898623832s" totalImagesPullingTime="4.486737997s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:13:57.898108122 +0000 UTC m=+71.752614925" watchObservedRunningTime="2026-06-08 18:13:57.898623832 +0000 UTC m=+71.753130665" Jun 08 18:13:57.914137 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:13:57.914092 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kznkg" podStartSLOduration=67.448378943 podStartE2EDuration="1m11.914078295s" totalImagesPullingTime="4.465699352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:13:57.913245929 +0000 UTC m=+71.767752728" watchObservedRunningTime="2026-06-08 18:13:57.914078295 +0000 UTC m=+71.768585093" Jun 08 18:14:24.862071 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:14:24.862039 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls\") pod \"dns-default-n4m82\" (UID: \"c2665ffe-cccc-432a-85a7-a5244f0b2bc8\") " pod="openshift-dns/dns-default-n4m82" Jun 08 18:14:24.862405 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:14:24.862146 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 18:14:24.862405 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:14:24.862210 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls podName:c2665ffe-cccc-432a-85a7-a5244f0b2bc8 nodeName:}" failed. No retries permitted until 2026-06-08 18:15:28.862194926 +0000 UTC m=+162.716701702 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls") pod "dns-default-n4m82" (UID: "c2665ffe-cccc-432a-85a7-a5244f0b2bc8") : secret "dns-default-metrics-tls" not found Jun 08 18:14:28.886259 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:14:28.886231 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kznkg" Jun 08 18:14:56.372138 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:14:56.372088 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs\") pod \"network-metrics-daemon-jjs4l\" (UID: \"086390f2-8eb2-4903-a324-bf54451c435a\") " pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:14:56.372652 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:14:56.372243 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 08 18:14:56.372652 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:14:56.372334 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs podName:086390f2-8eb2-4903-a324-bf54451c435a nodeName:}" failed. No retries permitted until 2026-06-08 18:16:58.372312897 +0000 UTC m=+252.226819690 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs") pod "network-metrics-daemon-jjs4l" (UID: "086390f2-8eb2-4903-a324-bf54451c435a") : secret "metrics-daemon-secret" not found Jun 08 18:15:24.034632 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:15:24.034586 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-n4m82" podUID="c2665ffe-cccc-432a-85a7-a5244f0b2bc8" Jun 08 18:15:24.083739 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:15:24.083711 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n4m82" Jun 08 18:15:24.596050 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:15:24.596010 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-jjs4l" podUID="086390f2-8eb2-4903-a324-bf54451c435a" Jun 08 18:15:26.090374 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:15:26.090333 2572 generic.go:356] "Generic (PLEG): container finished" podID="7fcdb8ae-ba39-465c-9112-3f39e2e9032a" containerID="19c7d7dfda0ae963924f283752a1de51a965c9284302eca298dad99fd101dc9a" exitCode=255 Jun 08 18:15:26.090374 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:15:26.090376 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65584f7cd6-whmlg" event={"ID":"7fcdb8ae-ba39-465c-9112-3f39e2e9032a","Type":"ContainerDied","Data":"19c7d7dfda0ae963924f283752a1de51a965c9284302eca298dad99fd101dc9a"} Jun 08 18:15:26.095868 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:15:26.095847 2572 scope.go:117] "RemoveContainer" containerID="19c7d7dfda0ae963924f283752a1de51a965c9284302eca298dad99fd101dc9a" Jun 08 18:15:27.095482 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:15:27.095451 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65584f7cd6-whmlg" event={"ID":"7fcdb8ae-ba39-465c-9112-3f39e2e9032a","Type":"ContainerStarted","Data":"23cafd382ec1eb3680663110f9c8e5e8e08ae9801b5a1433924c481784dcce0c"} Jun 08 18:15:28.890042 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:15:28.890007 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls\") pod \"dns-default-n4m82\" (UID: \"c2665ffe-cccc-432a-85a7-a5244f0b2bc8\") " pod="openshift-dns/dns-default-n4m82" Jun 08 18:15:28.890440 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:15:28.890127 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 18:15:28.890440 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:15:28.890178 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls podName:c2665ffe-cccc-432a-85a7-a5244f0b2bc8 nodeName:}" failed. No retries permitted until 2026-06-08 18:17:30.890163088 +0000 UTC m=+284.744669864 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls") pod "dns-default-n4m82" (UID: "c2665ffe-cccc-432a-85a7-a5244f0b2bc8") : secret "dns-default-metrics-tls" not found Jun 08 18:15:38.587167 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:15:38.587073 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:16:00.782222 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.782060 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-lnhdg"] Jun 08 18:16:00.784087 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.784069 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-84f4b4799d-455f8"] Jun 08 18:16:00.784228 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.784207 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-lnhdg" Jun 08 18:16:00.785833 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.785815 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:00.787930 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.787909 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jun 08 18:16:00.788683 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.788670 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-dqwk6\"" Jun 08 18:16:00.789119 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.789105 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 08 18:16:00.791167 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.791145 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jun 08 18:16:00.791277 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:00.791146 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"registry-dockercfg-4mfgh\" is forbidden: User \"system:node:ip-10-0-138-194.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-138-194.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4mfgh\"" type="*v1.Secret" Jun 08 18:16:00.791339 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.791293 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 08 18:16:00.795667 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.795649 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 08 18:16:00.800908 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.800780 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-pkbdf"] Jun 08 18:16:00.803019 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.803002 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-pkbdf" Jun 08 18:16:00.807296 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.807273 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jun 08 18:16:00.807414 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.807299 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jun 08 18:16:00.807414 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.807372 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jun 08 18:16:00.807727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.807698 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-6j6b8\"" Jun 08 18:16:00.810018 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.810000 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 08 18:16:00.817785 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.817761 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-lnhdg"] Jun 08 18:16:00.819084 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.819066 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7d8bbf5d9c-f27lp"] Jun 08 18:16:00.821278 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.821261 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:00.824397 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.824382 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 08 18:16:00.824495 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.824411 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 08 18:16:00.824736 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.824719 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-6hjgn\"" Jun 08 18:16:00.824844 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.824825 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 08 18:16:00.826920 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.826900 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 08 18:16:00.827038 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.826983 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 08 18:16:00.830753 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.830732 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 08 18:16:00.833904 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.833876 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-pkbdf"] Jun 08 18:16:00.858352 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.858325 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84f4b4799d-455f8"] Jun 08 18:16:00.859054 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.859032 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7d8bbf5d9c-f27lp"] Jun 08 18:16:00.871201 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.871179 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-fl7lq"] Jun 08 18:16:00.872982 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.872947 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-fl7lq" Jun 08 18:16:00.874842 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.874795 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-wt66w\"" Jun 08 18:16:00.874981 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.874953 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jun 08 18:16:00.875040 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.875002 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jun 08 18:16:00.875040 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.874970 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jun 08 18:16:00.875260 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.875247 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jun 08 18:16:00.887719 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.887698 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-fl7lq"] Jun 08 18:16:00.902945 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.902921 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-certificates\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:00.903062 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.902967 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-stats-auth\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:00.903062 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.902990 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-metrics-certs\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:00.903062 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.903009 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb8jh\" (UniqueName: \"kubernetes.io/projected/65b0be8f-0667-45d6-a2e2-3ed94c8ee496-kube-api-access-mb8jh\") pod \"cluster-samples-operator-854f8b96b6-pkbdf\" (UID: \"65b0be8f-0667-45d6-a2e2-3ed94c8ee496\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-pkbdf" Jun 08 18:16:00.903062 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.903036 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75qlb\" (UniqueName: \"kubernetes.io/projected/685e569f-65a8-4c9b-b402-dc7ce33102ff-kube-api-access-75qlb\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:00.903209 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.903097 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/79eae03b-1bf3-40dd-adfd-be1fd34224ae-ca-trust-extracted\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:00.903209 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.903119 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/79eae03b-1bf3-40dd-adfd-be1fd34224ae-installation-pull-secrets\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:00.903209 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.903141 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/65b0be8f-0667-45d6-a2e2-3ed94c8ee496-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-pkbdf\" (UID: \"65b0be8f-0667-45d6-a2e2-3ed94c8ee496\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-pkbdf" Jun 08 18:16:00.903209 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.903158 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjknf\" (UniqueName: \"kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-kube-api-access-rjknf\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:00.903358 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.903209 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/79eae03b-1bf3-40dd-adfd-be1fd34224ae-image-registry-private-configuration\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:00.903358 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.903234 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-bound-sa-token\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:00.903358 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.903252 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-default-certificate\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:00.903358 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.903271 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5g8c\" (UniqueName: \"kubernetes.io/projected/7782276f-b077-46b8-977e-2e7855e77764-kube-api-access-n5g8c\") pod \"volume-data-source-validator-78dd696465-lnhdg\" (UID: \"7782276f-b077-46b8-977e-2e7855e77764\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-lnhdg" Jun 08 18:16:00.903358 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.903287 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/79eae03b-1bf3-40dd-adfd-be1fd34224ae-trusted-ca\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:00.903358 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.903302 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/685e569f-65a8-4c9b-b402-dc7ce33102ff-service-ca-bundle\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:00.903358 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:00.903320 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-tls\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:01.004033 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.003994 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjknf\" (UniqueName: \"kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-kube-api-access-rjknf\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:01.004033 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.004036 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/79eae03b-1bf3-40dd-adfd-be1fd34224ae-ca-trust-extracted\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:01.004290 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.004060 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/65b0be8f-0667-45d6-a2e2-3ed94c8ee496-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-pkbdf\" (UID: \"65b0be8f-0667-45d6-a2e2-3ed94c8ee496\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-pkbdf" Jun 08 18:16:01.004290 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.004088 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/62df564d-0c6d-4fd1-a67c-de8d9dc9d5a7-config\") pod \"service-ca-operator-6bc5ff68f5-fl7lq\" (UID: \"62df564d-0c6d-4fd1-a67c-de8d9dc9d5a7\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-fl7lq" Jun 08 18:16:01.004290 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.004127 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-default-certificate\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:01.004290 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.004151 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5g8c\" (UniqueName: \"kubernetes.io/projected/7782276f-b077-46b8-977e-2e7855e77764-kube-api-access-n5g8c\") pod \"volume-data-source-validator-78dd696465-lnhdg\" (UID: \"7782276f-b077-46b8-977e-2e7855e77764\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-lnhdg" Jun 08 18:16:01.004290 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:01.004221 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 08 18:16:01.004290 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.004265 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/79eae03b-1bf3-40dd-adfd-be1fd34224ae-trusted-ca\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:01.004595 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:01.004302 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65b0be8f-0667-45d6-a2e2-3ed94c8ee496-samples-operator-tls podName:65b0be8f-0667-45d6-a2e2-3ed94c8ee496 nodeName:}" failed. No retries permitted until 2026-06-08 18:16:01.504280311 +0000 UTC m=+195.358787092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/65b0be8f-0667-45d6-a2e2-3ed94c8ee496-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-pkbdf" (UID: "65b0be8f-0667-45d6-a2e2-3ed94c8ee496") : secret "samples-operator-tls" not found Jun 08 18:16:01.004595 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.004336 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-tls\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:01.004595 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.004369 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-certificates\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:01.004595 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.004396 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-metrics-certs\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:01.004595 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.004415 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/79eae03b-1bf3-40dd-adfd-be1fd34224ae-ca-trust-extracted\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:01.004595 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.004423 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb8jh\" (UniqueName: \"kubernetes.io/projected/65b0be8f-0667-45d6-a2e2-3ed94c8ee496-kube-api-access-mb8jh\") pod \"cluster-samples-operator-854f8b96b6-pkbdf\" (UID: \"65b0be8f-0667-45d6-a2e2-3ed94c8ee496\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-pkbdf" Jun 08 18:16:01.004595 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:01.004449 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 18:16:01.004595 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:01.004464 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f4b4799d-455f8: secret "image-registry-tls" not found Jun 08 18:16:01.004595 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.004479 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75qlb\" (UniqueName: \"kubernetes.io/projected/685e569f-65a8-4c9b-b402-dc7ce33102ff-kube-api-access-75qlb\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:01.004595 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:01.004523 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-tls podName:79eae03b-1bf3-40dd-adfd-be1fd34224ae nodeName:}" failed. No retries permitted until 2026-06-08 18:16:01.504503917 +0000 UTC m=+195.359010697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-tls") pod "image-registry-84f4b4799d-455f8" (UID: "79eae03b-1bf3-40dd-adfd-be1fd34224ae") : secret "image-registry-tls" not found Jun 08 18:16:01.004595 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:01.004531 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 08 18:16:01.004595 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.004567 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/79eae03b-1bf3-40dd-adfd-be1fd34224ae-installation-pull-secrets\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:01.004595 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:01.004584 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-metrics-certs podName:685e569f-65a8-4c9b-b402-dc7ce33102ff nodeName:}" failed. No retries permitted until 2026-06-08 18:16:01.504569093 +0000 UTC m=+195.359075882 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-metrics-certs") pod "router-default-7d8bbf5d9c-f27lp" (UID: "685e569f-65a8-4c9b-b402-dc7ce33102ff") : secret "router-metrics-certs-default" not found Jun 08 18:16:01.005277 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.004619 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2jch\" (UniqueName: \"kubernetes.io/projected/62df564d-0c6d-4fd1-a67c-de8d9dc9d5a7-kube-api-access-h2jch\") pod \"service-ca-operator-6bc5ff68f5-fl7lq\" (UID: \"62df564d-0c6d-4fd1-a67c-de8d9dc9d5a7\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-fl7lq" Jun 08 18:16:01.005277 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.004665 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-stats-auth\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:01.005277 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.004697 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-bound-sa-token\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:01.005277 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.004724 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/79eae03b-1bf3-40dd-adfd-be1fd34224ae-image-registry-private-configuration\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:01.005277 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.004755 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/685e569f-65a8-4c9b-b402-dc7ce33102ff-service-ca-bundle\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:01.005277 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.004787 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/62df564d-0c6d-4fd1-a67c-de8d9dc9d5a7-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-fl7lq\" (UID: \"62df564d-0c6d-4fd1-a67c-de8d9dc9d5a7\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-fl7lq" Jun 08 18:16:01.005277 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.004995 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-certificates\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:01.005277 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:01.005032 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/685e569f-65a8-4c9b-b402-dc7ce33102ff-service-ca-bundle podName:685e569f-65a8-4c9b-b402-dc7ce33102ff nodeName:}" failed. No retries permitted until 2026-06-08 18:16:01.505015296 +0000 UTC m=+195.359522073 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/685e569f-65a8-4c9b-b402-dc7ce33102ff-service-ca-bundle") pod "router-default-7d8bbf5d9c-f27lp" (UID: "685e569f-65a8-4c9b-b402-dc7ce33102ff") : configmap references non-existent config key: service-ca.crt Jun 08 18:16:01.005677 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.005414 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/79eae03b-1bf3-40dd-adfd-be1fd34224ae-trusted-ca\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:01.006899 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.006876 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-default-certificate\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:01.007070 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.007053 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-stats-auth\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:01.007215 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.007198 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/79eae03b-1bf3-40dd-adfd-be1fd34224ae-installation-pull-secrets\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:01.007362 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.007341 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/79eae03b-1bf3-40dd-adfd-be1fd34224ae-image-registry-private-configuration\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:01.025154 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.025126 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5g8c\" (UniqueName: \"kubernetes.io/projected/7782276f-b077-46b8-977e-2e7855e77764-kube-api-access-n5g8c\") pod \"volume-data-source-validator-78dd696465-lnhdg\" (UID: \"7782276f-b077-46b8-977e-2e7855e77764\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-lnhdg" Jun 08 18:16:01.025254 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.025179 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-bound-sa-token\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:01.025254 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.025238 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjknf\" (UniqueName: \"kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-kube-api-access-rjknf\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:01.033308 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.033252 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb8jh\" (UniqueName: \"kubernetes.io/projected/65b0be8f-0667-45d6-a2e2-3ed94c8ee496-kube-api-access-mb8jh\") pod \"cluster-samples-operator-854f8b96b6-pkbdf\" (UID: \"65b0be8f-0667-45d6-a2e2-3ed94c8ee496\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-pkbdf" Jun 08 18:16:01.034682 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.034664 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75qlb\" (UniqueName: \"kubernetes.io/projected/685e569f-65a8-4c9b-b402-dc7ce33102ff-kube-api-access-75qlb\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:01.093721 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.093684 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-lnhdg" Jun 08 18:16:01.105371 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.105342 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/62df564d-0c6d-4fd1-a67c-de8d9dc9d5a7-config\") pod \"service-ca-operator-6bc5ff68f5-fl7lq\" (UID: \"62df564d-0c6d-4fd1-a67c-de8d9dc9d5a7\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-fl7lq" Jun 08 18:16:01.105480 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.105453 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2jch\" (UniqueName: \"kubernetes.io/projected/62df564d-0c6d-4fd1-a67c-de8d9dc9d5a7-kube-api-access-h2jch\") pod \"service-ca-operator-6bc5ff68f5-fl7lq\" (UID: \"62df564d-0c6d-4fd1-a67c-de8d9dc9d5a7\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-fl7lq" Jun 08 18:16:01.105534 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.105505 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/62df564d-0c6d-4fd1-a67c-de8d9dc9d5a7-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-fl7lq\" (UID: \"62df564d-0c6d-4fd1-a67c-de8d9dc9d5a7\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-fl7lq" Jun 08 18:16:01.105975 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.105938 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/62df564d-0c6d-4fd1-a67c-de8d9dc9d5a7-config\") pod \"service-ca-operator-6bc5ff68f5-fl7lq\" (UID: \"62df564d-0c6d-4fd1-a67c-de8d9dc9d5a7\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-fl7lq" Jun 08 18:16:01.107636 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.107617 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/62df564d-0c6d-4fd1-a67c-de8d9dc9d5a7-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-fl7lq\" (UID: \"62df564d-0c6d-4fd1-a67c-de8d9dc9d5a7\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-fl7lq" Jun 08 18:16:01.123933 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.123910 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2jch\" (UniqueName: \"kubernetes.io/projected/62df564d-0c6d-4fd1-a67c-de8d9dc9d5a7-kube-api-access-h2jch\") pod \"service-ca-operator-6bc5ff68f5-fl7lq\" (UID: \"62df564d-0c6d-4fd1-a67c-de8d9dc9d5a7\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-fl7lq" Jun 08 18:16:01.181244 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.181210 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-fl7lq" Jun 08 18:16:01.233619 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.233583 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-lnhdg"] Jun 08 18:16:01.236758 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:16:01.236724 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7782276f_b077_46b8_977e_2e7855e77764.slice/crio-cc4679fcf8dab910df42eaf7eae90fa63a5e47b7112218cf423ba7c5c0235421 WatchSource:0}: Error finding container cc4679fcf8dab910df42eaf7eae90fa63a5e47b7112218cf423ba7c5c0235421: Status 404 returned error can't find the container with id cc4679fcf8dab910df42eaf7eae90fa63a5e47b7112218cf423ba7c5c0235421 Jun 08 18:16:01.333524 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:16:01.333485 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62df564d_0c6d_4fd1_a67c_de8d9dc9d5a7.slice/crio-af6a8237c6c3707bf4db70b3bc9fb237756086eed1731fd3d87d64751d284cd9 WatchSource:0}: Error finding container af6a8237c6c3707bf4db70b3bc9fb237756086eed1731fd3d87d64751d284cd9: Status 404 returned error can't find the container with id af6a8237c6c3707bf4db70b3bc9fb237756086eed1731fd3d87d64751d284cd9 Jun 08 18:16:01.334484 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.334450 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-fl7lq"] Jun 08 18:16:01.508444 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.508405 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-metrics-certs\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:01.508617 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.508472 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/685e569f-65a8-4c9b-b402-dc7ce33102ff-service-ca-bundle\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:01.508617 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.508506 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/65b0be8f-0667-45d6-a2e2-3ed94c8ee496-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-pkbdf\" (UID: \"65b0be8f-0667-45d6-a2e2-3ed94c8ee496\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-pkbdf" Jun 08 18:16:01.508617 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.508545 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-tls\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:01.508617 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:01.508555 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 08 18:16:01.508770 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:01.508625 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 08 18:16:01.508770 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:01.508650 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 18:16:01.508770 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:01.508662 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f4b4799d-455f8: secret "image-registry-tls" not found Jun 08 18:16:01.508770 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:01.508632 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-metrics-certs podName:685e569f-65a8-4c9b-b402-dc7ce33102ff nodeName:}" failed. No retries permitted until 2026-06-08 18:16:02.508611422 +0000 UTC m=+196.363118199 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-metrics-certs") pod "router-default-7d8bbf5d9c-f27lp" (UID: "685e569f-65a8-4c9b-b402-dc7ce33102ff") : secret "router-metrics-certs-default" not found Jun 08 18:16:01.508770 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:01.508691 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/685e569f-65a8-4c9b-b402-dc7ce33102ff-service-ca-bundle podName:685e569f-65a8-4c9b-b402-dc7ce33102ff nodeName:}" failed. No retries permitted until 2026-06-08 18:16:02.508678912 +0000 UTC m=+196.363185696 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/685e569f-65a8-4c9b-b402-dc7ce33102ff-service-ca-bundle") pod "router-default-7d8bbf5d9c-f27lp" (UID: "685e569f-65a8-4c9b-b402-dc7ce33102ff") : configmap references non-existent config key: service-ca.crt Jun 08 18:16:01.508770 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:01.508705 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65b0be8f-0667-45d6-a2e2-3ed94c8ee496-samples-operator-tls podName:65b0be8f-0667-45d6-a2e2-3ed94c8ee496 nodeName:}" failed. No retries permitted until 2026-06-08 18:16:02.508697354 +0000 UTC m=+196.363204143 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/65b0be8f-0667-45d6-a2e2-3ed94c8ee496-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-pkbdf" (UID: "65b0be8f-0667-45d6-a2e2-3ed94c8ee496") : secret "samples-operator-tls" not found Jun 08 18:16:01.508770 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:01.508727 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-tls podName:79eae03b-1bf3-40dd-adfd-be1fd34224ae nodeName:}" failed. No retries permitted until 2026-06-08 18:16:02.508717913 +0000 UTC m=+196.363224693 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-tls") pod "image-registry-84f4b4799d-455f8" (UID: "79eae03b-1bf3-40dd-adfd-be1fd34224ae") : secret "image-registry-tls" not found Jun 08 18:16:01.922676 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:01.922644 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4mfgh\"" Jun 08 18:16:02.189712 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:02.189616 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-fl7lq" event={"ID":"62df564d-0c6d-4fd1-a67c-de8d9dc9d5a7","Type":"ContainerStarted","Data":"af6a8237c6c3707bf4db70b3bc9fb237756086eed1731fd3d87d64751d284cd9"} Jun 08 18:16:02.190753 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:02.190728 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-lnhdg" event={"ID":"7782276f-b077-46b8-977e-2e7855e77764","Type":"ContainerStarted","Data":"cc4679fcf8dab910df42eaf7eae90fa63a5e47b7112218cf423ba7c5c0235421"} Jun 08 18:16:02.517001 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:02.516952 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-metrics-certs\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:02.517113 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:02.517062 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/685e569f-65a8-4c9b-b402-dc7ce33102ff-service-ca-bundle\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:02.517113 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:02.517090 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 08 18:16:02.517229 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:02.517178 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 08 18:16:02.517229 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:02.517100 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/65b0be8f-0667-45d6-a2e2-3ed94c8ee496-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-pkbdf\" (UID: \"65b0be8f-0667-45d6-a2e2-3ed94c8ee496\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-pkbdf" Jun 08 18:16:02.517229 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:02.517226 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65b0be8f-0667-45d6-a2e2-3ed94c8ee496-samples-operator-tls podName:65b0be8f-0667-45d6-a2e2-3ed94c8ee496 nodeName:}" failed. No retries permitted until 2026-06-08 18:16:04.517211701 +0000 UTC m=+198.371718477 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/65b0be8f-0667-45d6-a2e2-3ed94c8ee496-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-pkbdf" (UID: "65b0be8f-0667-45d6-a2e2-3ed94c8ee496") : secret "samples-operator-tls" not found Jun 08 18:16:02.517383 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:02.517313 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/685e569f-65a8-4c9b-b402-dc7ce33102ff-service-ca-bundle podName:685e569f-65a8-4c9b-b402-dc7ce33102ff nodeName:}" failed. No retries permitted until 2026-06-08 18:16:04.517291559 +0000 UTC m=+198.371798349 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/685e569f-65a8-4c9b-b402-dc7ce33102ff-service-ca-bundle") pod "router-default-7d8bbf5d9c-f27lp" (UID: "685e569f-65a8-4c9b-b402-dc7ce33102ff") : configmap references non-existent config key: service-ca.crt Jun 08 18:16:02.517383 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:02.517348 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-metrics-certs podName:685e569f-65a8-4c9b-b402-dc7ce33102ff nodeName:}" failed. No retries permitted until 2026-06-08 18:16:04.517335632 +0000 UTC m=+198.371842408 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-metrics-certs") pod "router-default-7d8bbf5d9c-f27lp" (UID: "685e569f-65a8-4c9b-b402-dc7ce33102ff") : secret "router-metrics-certs-default" not found Jun 08 18:16:02.517495 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:02.517432 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-tls\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:02.517559 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:02.517545 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 18:16:02.517608 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:02.517563 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f4b4799d-455f8: secret "image-registry-tls" not found Jun 08 18:16:02.517608 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:02.517607 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-tls podName:79eae03b-1bf3-40dd-adfd-be1fd34224ae nodeName:}" failed. No retries permitted until 2026-06-08 18:16:04.517593976 +0000 UTC m=+198.372100769 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-tls") pod "image-registry-84f4b4799d-455f8" (UID: "79eae03b-1bf3-40dd-adfd-be1fd34224ae") : secret "image-registry-tls" not found Jun 08 18:16:03.197019 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:03.196981 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-lnhdg" event={"ID":"7782276f-b077-46b8-977e-2e7855e77764","Type":"ContainerStarted","Data":"7af1425f96407082a1c02a69145fbab52c7276842ecfa3bb824b1edd028239b5"} Jun 08 18:16:03.212920 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:03.212847 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-lnhdg" podStartSLOduration=1.96416985 podStartE2EDuration="3.212824872s" totalImagesPullingTime="1.248655022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:16:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:16:03.212768708 +0000 UTC m=+197.067275507" watchObservedRunningTime="2026-06-08 18:16:03.212824872 +0000 UTC m=+197.067331672" Jun 08 18:16:04.201854 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:04.201817 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-fl7lq" event={"ID":"62df564d-0c6d-4fd1-a67c-de8d9dc9d5a7","Type":"ContainerStarted","Data":"41df78ea43fcefe7c6facd234665c65ea3a52d1677cd29f0a58b346dd76ef119"} Jun 08 18:16:04.217684 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:04.217633 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-fl7lq" podStartSLOduration=2.100944803 podStartE2EDuration="4.217618879s" totalImagesPullingTime="2.116674076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:16:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:16:04.216679547 +0000 UTC m=+198.071186346" watchObservedRunningTime="2026-06-08 18:16:04.217618879 +0000 UTC m=+198.072125676" Jun 08 18:16:04.533624 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:04.533525 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/65b0be8f-0667-45d6-a2e2-3ed94c8ee496-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-pkbdf\" (UID: \"65b0be8f-0667-45d6-a2e2-3ed94c8ee496\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-pkbdf" Jun 08 18:16:04.533624 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:04.533589 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-tls\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:04.533624 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:04.533614 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-metrics-certs\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:04.533852 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:04.533655 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/685e569f-65a8-4c9b-b402-dc7ce33102ff-service-ca-bundle\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:04.533852 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:04.533679 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 08 18:16:04.533852 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:04.533733 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 18:16:04.533852 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:04.533740 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 08 18:16:04.533852 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:04.533749 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f4b4799d-455f8: secret "image-registry-tls" not found Jun 08 18:16:04.533852 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:04.533764 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65b0be8f-0667-45d6-a2e2-3ed94c8ee496-samples-operator-tls podName:65b0be8f-0667-45d6-a2e2-3ed94c8ee496 nodeName:}" failed. No retries permitted until 2026-06-08 18:16:08.533743641 +0000 UTC m=+202.388250439 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/65b0be8f-0667-45d6-a2e2-3ed94c8ee496-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-pkbdf" (UID: "65b0be8f-0667-45d6-a2e2-3ed94c8ee496") : secret "samples-operator-tls" not found Jun 08 18:16:04.533852 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:04.533782 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/685e569f-65a8-4c9b-b402-dc7ce33102ff-service-ca-bundle podName:685e569f-65a8-4c9b-b402-dc7ce33102ff nodeName:}" failed. No retries permitted until 2026-06-08 18:16:08.533773818 +0000 UTC m=+202.388280598 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/685e569f-65a8-4c9b-b402-dc7ce33102ff-service-ca-bundle") pod "router-default-7d8bbf5d9c-f27lp" (UID: "685e569f-65a8-4c9b-b402-dc7ce33102ff") : configmap references non-existent config key: service-ca.crt Jun 08 18:16:04.533852 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:04.533795 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-metrics-certs podName:685e569f-65a8-4c9b-b402-dc7ce33102ff nodeName:}" failed. No retries permitted until 2026-06-08 18:16:08.533788838 +0000 UTC m=+202.388295614 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-metrics-certs") pod "router-default-7d8bbf5d9c-f27lp" (UID: "685e569f-65a8-4c9b-b402-dc7ce33102ff") : secret "router-metrics-certs-default" not found Jun 08 18:16:04.533852 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:04.533807 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-tls podName:79eae03b-1bf3-40dd-adfd-be1fd34224ae nodeName:}" failed. No retries permitted until 2026-06-08 18:16:08.533800767 +0000 UTC m=+202.388307547 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-tls") pod "image-registry-84f4b4799d-455f8" (UID: "79eae03b-1bf3-40dd-adfd-be1fd34224ae") : secret "image-registry-tls" not found Jun 08 18:16:08.569902 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:08.569864 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-tls\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:08.569902 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:08.569907 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-metrics-certs\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:08.570395 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:08.569951 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/685e569f-65a8-4c9b-b402-dc7ce33102ff-service-ca-bundle\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:08.570395 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:08.570000 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/65b0be8f-0667-45d6-a2e2-3ed94c8ee496-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-pkbdf\" (UID: \"65b0be8f-0667-45d6-a2e2-3ed94c8ee496\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-pkbdf" Jun 08 18:16:08.570395 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:08.570072 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 18:16:08.570395 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:08.570089 2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 08 18:16:08.570395 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:08.570097 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f4b4799d-455f8: secret "image-registry-tls" not found Jun 08 18:16:08.570395 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:08.570115 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/685e569f-65a8-4c9b-b402-dc7ce33102ff-service-ca-bundle podName:685e569f-65a8-4c9b-b402-dc7ce33102ff nodeName:}" failed. No retries permitted until 2026-06-08 18:16:16.570098556 +0000 UTC m=+210.424605335 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/685e569f-65a8-4c9b-b402-dc7ce33102ff-service-ca-bundle") pod "router-default-7d8bbf5d9c-f27lp" (UID: "685e569f-65a8-4c9b-b402-dc7ce33102ff") : configmap references non-existent config key: service-ca.crt Jun 08 18:16:08.570395 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:08.570084 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 08 18:16:08.570395 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:08.570135 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65b0be8f-0667-45d6-a2e2-3ed94c8ee496-samples-operator-tls podName:65b0be8f-0667-45d6-a2e2-3ed94c8ee496 nodeName:}" failed. No retries permitted until 2026-06-08 18:16:16.570127615 +0000 UTC m=+210.424634392 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/65b0be8f-0667-45d6-a2e2-3ed94c8ee496-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-pkbdf" (UID: "65b0be8f-0667-45d6-a2e2-3ed94c8ee496") : secret "samples-operator-tls" not found Jun 08 18:16:08.570395 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:08.570148 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-tls podName:79eae03b-1bf3-40dd-adfd-be1fd34224ae nodeName:}" failed. No retries permitted until 2026-06-08 18:16:16.570141636 +0000 UTC m=+210.424648413 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-tls") pod "image-registry-84f4b4799d-455f8" (UID: "79eae03b-1bf3-40dd-adfd-be1fd34224ae") : secret "image-registry-tls" not found Jun 08 18:16:08.570395 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:08.570170 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-metrics-certs podName:685e569f-65a8-4c9b-b402-dc7ce33102ff nodeName:}" failed. No retries permitted until 2026-06-08 18:16:16.570152455 +0000 UTC m=+210.424659245 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-metrics-certs") pod "router-default-7d8bbf5d9c-f27lp" (UID: "685e569f-65a8-4c9b-b402-dc7ce33102ff") : secret "router-metrics-certs-default" not found Jun 08 18:16:08.721389 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:08.721360 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z77dq_1ad02886-8d99-4ad6-8103-a3eafea4af48/dns-node-resolver/0.log" Jun 08 18:16:09.510560 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:09.510531 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bvg6f_445f7d76-73b2-4d8c-aad9-9195be1bd5fa/node-ca/0.log" Jun 08 18:16:10.111159 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:10.111123 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bl784_0a6fe03e-3966-431d-99ff-43a27c4d0249/serve-healthcheck-canary/0.log" Jun 08 18:16:16.636380 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:16.636342 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-tls\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:16.636853 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:16.636388 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-metrics-certs\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:16.636853 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:16.636442 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/685e569f-65a8-4c9b-b402-dc7ce33102ff-service-ca-bundle\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:16.636853 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:16.636514 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/65b0be8f-0667-45d6-a2e2-3ed94c8ee496-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-pkbdf\" (UID: \"65b0be8f-0667-45d6-a2e2-3ed94c8ee496\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-pkbdf" Jun 08 18:16:16.636853 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:16:16.636662 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/685e569f-65a8-4c9b-b402-dc7ce33102ff-service-ca-bundle podName:685e569f-65a8-4c9b-b402-dc7ce33102ff nodeName:}" failed. No retries permitted until 2026-06-08 18:16:32.636644422 +0000 UTC m=+226.491151217 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/685e569f-65a8-4c9b-b402-dc7ce33102ff-service-ca-bundle") pod "router-default-7d8bbf5d9c-f27lp" (UID: "685e569f-65a8-4c9b-b402-dc7ce33102ff") : configmap references non-existent config key: service-ca.crt Jun 08 18:16:16.638780 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:16.638753 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-tls\") pod \"image-registry-84f4b4799d-455f8\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:16.638780 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:16.638768 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/65b0be8f-0667-45d6-a2e2-3ed94c8ee496-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-pkbdf\" (UID: \"65b0be8f-0667-45d6-a2e2-3ed94c8ee496\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-pkbdf" Jun 08 18:16:16.638922 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:16.638841 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/685e569f-65a8-4c9b-b402-dc7ce33102ff-metrics-certs\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:16.699765 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:16.699737 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:16.710657 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:16.710630 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-pkbdf" Jun 08 18:16:16.845622 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:16.845588 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84f4b4799d-455f8"] Jun 08 18:16:16.848372 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:16:16.848346 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79eae03b_1bf3_40dd_adfd_be1fd34224ae.slice/crio-fa1e775603938d9c8b2fbcd11e88b7d5ceb1ffee7638a197607549183386fcac WatchSource:0}: Error finding container fa1e775603938d9c8b2fbcd11e88b7d5ceb1ffee7638a197607549183386fcac: Status 404 returned error can't find the container with id fa1e775603938d9c8b2fbcd11e88b7d5ceb1ffee7638a197607549183386fcac Jun 08 18:16:16.852676 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:16.852654 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-pkbdf"] Jun 08 18:16:17.241856 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:17.241819 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84f4b4799d-455f8" event={"ID":"79eae03b-1bf3-40dd-adfd-be1fd34224ae","Type":"ContainerStarted","Data":"a3c78e9036ef7d96a0532ce18fc1d5ce736f5b5d80c44532eeaee25a51a1ad74"} Jun 08 18:16:17.242101 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:17.241896 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84f4b4799d-455f8" event={"ID":"79eae03b-1bf3-40dd-adfd-be1fd34224ae","Type":"ContainerStarted","Data":"fa1e775603938d9c8b2fbcd11e88b7d5ceb1ffee7638a197607549183386fcac"} Jun 08 18:16:17.242101 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:17.241947 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:17.243753 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:17.243713 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-pkbdf" event={"ID":"65b0be8f-0667-45d6-a2e2-3ed94c8ee496","Type":"ContainerStarted","Data":"20c30ac42ae1a552df2bc078565f37274a63b94849bf44b25130d97e1e04214b"} Jun 08 18:16:17.271527 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:17.271471 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-84f4b4799d-455f8" podStartSLOduration=17.27145397 podStartE2EDuration="17.27145397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:16:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:16:17.270891679 +0000 UTC m=+211.125398477" watchObservedRunningTime="2026-06-08 18:16:17.27145397 +0000 UTC m=+211.125960767" Jun 08 18:16:19.251812 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:19.251732 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-pkbdf" event={"ID":"65b0be8f-0667-45d6-a2e2-3ed94c8ee496","Type":"ContainerStarted","Data":"ae27a0ec0a122bcd70c28705471fcec385dfdb11e3817c8d49803a3ffee9d261"} Jun 08 18:16:19.251812 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:19.251791 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-pkbdf" event={"ID":"65b0be8f-0667-45d6-a2e2-3ed94c8ee496","Type":"ContainerStarted","Data":"fba504b7004dc2bf774559d234c4f1368b14426f3f4d66dbb47b55e51f35f53d"} Jun 08 18:16:19.274653 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:19.274599 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-pkbdf" podStartSLOduration=17.216981589 podStartE2EDuration="19.274581514s" totalImagesPullingTime="2.057599925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:16:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:16:19.274013979 +0000 UTC m=+213.128520776" watchObservedRunningTime="2026-06-08 18:16:19.274581514 +0000 UTC m=+213.129088350" Jun 08 18:16:32.242644 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.242609 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84f4b4799d-455f8"] Jun 08 18:16:32.247036 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.246992 2572 patch_prober.go:28] interesting pod/image-registry-84f4b4799d-455f8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 18:16:32.247201 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.247048 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-84f4b4799d-455f8" podUID="79eae03b-1bf3-40dd-adfd-be1fd34224ae" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 18:16:32.300069 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.300039 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-dc69d89dc-lwb8p"] Jun 08 18:16:32.304712 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.304691 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.318765 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.318740 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-dc69d89dc-lwb8p"] Jun 08 18:16:32.347589 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.347556 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g976j\" (UniqueName: \"kubernetes.io/projected/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-kube-api-access-g976j\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.347755 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.347626 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-trusted-ca\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.347755 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.347654 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-registry-certificates\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.347878 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.347777 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-installation-pull-secrets\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.347878 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.347819 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-registry-tls\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.347878 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.347848 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-bound-sa-token\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.347878 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.347874 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-ca-trust-extracted\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.348119 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.347921 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-image-registry-private-configuration\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.414867 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.414841 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gb9rc"] Jun 08 18:16:32.418807 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.418786 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gb9rc" Jun 08 18:16:32.420881 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.420852 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 08 18:16:32.421015 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.420876 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hgqqt\"" Jun 08 18:16:32.421081 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.421010 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 08 18:16:32.421150 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.421124 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 08 18:16:32.421207 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.421193 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 08 18:16:32.436113 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.436089 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gb9rc"] Jun 08 18:16:32.448748 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.448725 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-installation-pull-secrets\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.448862 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.448757 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-registry-tls\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.448862 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.448777 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca9bcf18-14c9-4280-babf-4466a27fe082-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gb9rc\" (UID: \"ca9bcf18-14c9-4280-babf-4466a27fe082\") " pod="openshift-insights/insights-runtime-extractor-gb9rc" Jun 08 18:16:32.448862 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.448798 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dckfm\" (UniqueName: \"kubernetes.io/projected/ca9bcf18-14c9-4280-babf-4466a27fe082-kube-api-access-dckfm\") pod \"insights-runtime-extractor-gb9rc\" (UID: \"ca9bcf18-14c9-4280-babf-4466a27fe082\") " pod="openshift-insights/insights-runtime-extractor-gb9rc" Jun 08 18:16:32.448862 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.448847 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-bound-sa-token\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.449100 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.448870 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-ca-trust-extracted\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.449100 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.448898 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-image-registry-private-configuration\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.449100 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.448933 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g976j\" (UniqueName: \"kubernetes.io/projected/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-kube-api-access-g976j\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.449100 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.448971 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ca9bcf18-14c9-4280-babf-4466a27fe082-crio-socket\") pod \"insights-runtime-extractor-gb9rc\" (UID: \"ca9bcf18-14c9-4280-babf-4466a27fe082\") " pod="openshift-insights/insights-runtime-extractor-gb9rc" Jun 08 18:16:32.449100 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.449004 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca9bcf18-14c9-4280-babf-4466a27fe082-data-volume\") pod \"insights-runtime-extractor-gb9rc\" (UID: \"ca9bcf18-14c9-4280-babf-4466a27fe082\") " pod="openshift-insights/insights-runtime-extractor-gb9rc" Jun 08 18:16:32.449100 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.449033 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-trusted-ca\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.449100 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.449062 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-registry-certificates\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.449434 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.449235 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca9bcf18-14c9-4280-babf-4466a27fe082-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gb9rc\" (UID: \"ca9bcf18-14c9-4280-babf-4466a27fe082\") " pod="openshift-insights/insights-runtime-extractor-gb9rc" Jun 08 18:16:32.449434 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.449292 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-ca-trust-extracted\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.449978 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.449933 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-trusted-ca\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.450084 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.450026 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-registry-certificates\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.451316 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.451294 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-image-registry-private-configuration\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.451316 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.451311 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-installation-pull-secrets\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.451468 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.451382 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-registry-tls\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.464833 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.464797 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-bound-sa-token\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.464924 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.464861 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g976j\" (UniqueName: \"kubernetes.io/projected/ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b-kube-api-access-g976j\") pod \"image-registry-dc69d89dc-lwb8p\" (UID: \"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b\") " pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.550486 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.550381 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ca9bcf18-14c9-4280-babf-4466a27fe082-crio-socket\") pod \"insights-runtime-extractor-gb9rc\" (UID: \"ca9bcf18-14c9-4280-babf-4466a27fe082\") " pod="openshift-insights/insights-runtime-extractor-gb9rc" Jun 08 18:16:32.550486 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.550431 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca9bcf18-14c9-4280-babf-4466a27fe082-data-volume\") pod \"insights-runtime-extractor-gb9rc\" (UID: \"ca9bcf18-14c9-4280-babf-4466a27fe082\") " pod="openshift-insights/insights-runtime-extractor-gb9rc" Jun 08 18:16:32.550701 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.550496 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca9bcf18-14c9-4280-babf-4466a27fe082-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gb9rc\" (UID: \"ca9bcf18-14c9-4280-babf-4466a27fe082\") " pod="openshift-insights/insights-runtime-extractor-gb9rc" Jun 08 18:16:32.550701 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.550517 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ca9bcf18-14c9-4280-babf-4466a27fe082-crio-socket\") pod \"insights-runtime-extractor-gb9rc\" (UID: \"ca9bcf18-14c9-4280-babf-4466a27fe082\") " pod="openshift-insights/insights-runtime-extractor-gb9rc" Jun 08 18:16:32.550701 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.550529 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca9bcf18-14c9-4280-babf-4466a27fe082-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gb9rc\" (UID: \"ca9bcf18-14c9-4280-babf-4466a27fe082\") " pod="openshift-insights/insights-runtime-extractor-gb9rc" Jun 08 18:16:32.550701 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.550634 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dckfm\" (UniqueName: \"kubernetes.io/projected/ca9bcf18-14c9-4280-babf-4466a27fe082-kube-api-access-dckfm\") pod \"insights-runtime-extractor-gb9rc\" (UID: \"ca9bcf18-14c9-4280-babf-4466a27fe082\") " pod="openshift-insights/insights-runtime-extractor-gb9rc" Jun 08 18:16:32.550898 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.550828 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca9bcf18-14c9-4280-babf-4466a27fe082-data-volume\") pod \"insights-runtime-extractor-gb9rc\" (UID: \"ca9bcf18-14c9-4280-babf-4466a27fe082\") " pod="openshift-insights/insights-runtime-extractor-gb9rc" Jun 08 18:16:32.551207 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.551186 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca9bcf18-14c9-4280-babf-4466a27fe082-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gb9rc\" (UID: \"ca9bcf18-14c9-4280-babf-4466a27fe082\") " pod="openshift-insights/insights-runtime-extractor-gb9rc" Jun 08 18:16:32.552815 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.552790 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca9bcf18-14c9-4280-babf-4466a27fe082-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gb9rc\" (UID: \"ca9bcf18-14c9-4280-babf-4466a27fe082\") " pod="openshift-insights/insights-runtime-extractor-gb9rc" Jun 08 18:16:32.559505 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.559483 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dckfm\" (UniqueName: \"kubernetes.io/projected/ca9bcf18-14c9-4280-babf-4466a27fe082-kube-api-access-dckfm\") pod \"insights-runtime-extractor-gb9rc\" (UID: \"ca9bcf18-14c9-4280-babf-4466a27fe082\") " pod="openshift-insights/insights-runtime-extractor-gb9rc" Jun 08 18:16:32.613690 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.613655 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:32.651695 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.651606 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/685e569f-65a8-4c9b-b402-dc7ce33102ff-service-ca-bundle\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:32.652363 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.652343 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/685e569f-65a8-4c9b-b402-dc7ce33102ff-service-ca-bundle\") pod \"router-default-7d8bbf5d9c-f27lp\" (UID: \"685e569f-65a8-4c9b-b402-dc7ce33102ff\") " pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:32.727640 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.727600 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gb9rc" Jun 08 18:16:32.776136 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.776105 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-dc69d89dc-lwb8p"] Jun 08 18:16:32.779137 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:16:32.779103 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebd76aeb_a1f1_4d59_b430_ac7d1eb6740b.slice/crio-91370eab6ac3c1d8aaf585de3ce91c3e4935d860e9bc41e2ff1811eb1ab83b05 WatchSource:0}: Error finding container 91370eab6ac3c1d8aaf585de3ce91c3e4935d860e9bc41e2ff1811eb1ab83b05: Status 404 returned error can't find the container with id 91370eab6ac3c1d8aaf585de3ce91c3e4935d860e9bc41e2ff1811eb1ab83b05 Jun 08 18:16:32.900781 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.900749 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gb9rc"] Jun 08 18:16:32.903930 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:16:32.903896 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca9bcf18_14c9_4280_babf_4466a27fe082.slice/crio-291756db0b7182bec205a10355b0e22d3bf61d837acf7081410a2ecbe7ab7ea3 WatchSource:0}: Error finding container 291756db0b7182bec205a10355b0e22d3bf61d837acf7081410a2ecbe7ab7ea3: Status 404 returned error can't find the container with id 291756db0b7182bec205a10355b0e22d3bf61d837acf7081410a2ecbe7ab7ea3 Jun 08 18:16:32.930359 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:32.930331 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:33.076266 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:33.076174 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7d8bbf5d9c-f27lp"] Jun 08 18:16:33.080255 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:16:33.080222 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod685e569f_65a8_4c9b_b402_dc7ce33102ff.slice/crio-c69f009ae3acbc76596189c70c67c8a1475ab99237facc3d6435ce3bde48adf6 WatchSource:0}: Error finding container c69f009ae3acbc76596189c70c67c8a1475ab99237facc3d6435ce3bde48adf6: Status 404 returned error can't find the container with id c69f009ae3acbc76596189c70c67c8a1475ab99237facc3d6435ce3bde48adf6 Jun 08 18:16:33.293192 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:33.293155 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gb9rc" event={"ID":"ca9bcf18-14c9-4280-babf-4466a27fe082","Type":"ContainerStarted","Data":"1687fb159782366791fd28c9dedd892f802d34b9c65102359c25101be411ce9b"} Jun 08 18:16:33.293587 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:33.293213 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gb9rc" event={"ID":"ca9bcf18-14c9-4280-babf-4466a27fe082","Type":"ContainerStarted","Data":"291756db0b7182bec205a10355b0e22d3bf61d837acf7081410a2ecbe7ab7ea3"} Jun 08 18:16:33.294665 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:33.294642 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" event={"ID":"685e569f-65a8-4c9b-b402-dc7ce33102ff","Type":"ContainerStarted","Data":"bfdae6288f9dd86320a66a897343caa6568f0ee0e1a4afbc0e09c43fc4aeccc8"} Jun 08 18:16:33.294791 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:33.294686 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" event={"ID":"685e569f-65a8-4c9b-b402-dc7ce33102ff","Type":"ContainerStarted","Data":"c69f009ae3acbc76596189c70c67c8a1475ab99237facc3d6435ce3bde48adf6"} Jun 08 18:16:33.300544 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:33.300512 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" event={"ID":"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b","Type":"ContainerStarted","Data":"e4e00b9ec541186714d38b6f4e7c0737a1e22b5e5a519894b0292c2c50344f06"} Jun 08 18:16:33.300677 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:33.300592 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" event={"ID":"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b","Type":"ContainerStarted","Data":"91370eab6ac3c1d8aaf585de3ce91c3e4935d860e9bc41e2ff1811eb1ab83b05"} Jun 08 18:16:33.300677 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:33.300655 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:16:33.315487 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:33.315351 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" podStartSLOduration=33.315333558 podStartE2EDuration="33.315333558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:16:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:16:33.314806118 +0000 UTC m=+227.169312917" watchObservedRunningTime="2026-06-08 18:16:33.315333558 +0000 UTC m=+227.169840357" Jun 08 18:16:33.333129 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:33.333047 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" podStartSLOduration=1.33303175 podStartE2EDuration="1.33303175s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:16:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:16:33.331765479 +0000 UTC m=+227.186272277" watchObservedRunningTime="2026-06-08 18:16:33.33303175 +0000 UTC m=+227.187538548" Jun 08 18:16:33.930772 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:33.930733 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:33.933257 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:33.933231 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:34.306743 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:34.306654 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gb9rc" event={"ID":"ca9bcf18-14c9-4280-babf-4466a27fe082","Type":"ContainerStarted","Data":"670ec0fcc905b7d8be875ea296fcd624edf3a3d56ad6422f246c0fb48932b9fd"} Jun 08 18:16:34.307174 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:34.306877 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:34.308166 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:34.308144 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7d8bbf5d9c-f27lp" Jun 08 18:16:36.317779 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:36.317740 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gb9rc" event={"ID":"ca9bcf18-14c9-4280-babf-4466a27fe082","Type":"ContainerStarted","Data":"3350cb5f7dc2ba5ed19a051831fd5214e39903c4660aa7e0b274a08b539aba2f"} Jun 08 18:16:36.337295 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:36.337246 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gb9rc" podStartSLOduration=2.114813779 podStartE2EDuration="4.33723127s" totalImagesPullingTime="2.222417491s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 18:16:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:16:36.335662457 +0000 UTC m=+230.190169266" watchObservedRunningTime="2026-06-08 18:16:36.33723127 +0000 UTC m=+230.191738069" Jun 08 18:16:42.247139 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:42.247106 2572 patch_prober.go:28] interesting pod/image-registry-84f4b4799d-455f8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 18:16:42.247529 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:42.247156 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-84f4b4799d-455f8" podUID="79eae03b-1bf3-40dd-adfd-be1fd34224ae" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 18:16:43.600420 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.600388 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-l8kvq"] Jun 08 18:16:43.605575 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.605558 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.608666 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.608639 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 08 18:16:43.608794 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.608649 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 08 18:16:43.609276 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.609257 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 08 18:16:43.609381 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.609311 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 08 18:16:43.609381 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.609328 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2hxdx\"" Jun 08 18:16:43.609381 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.609339 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 08 18:16:43.609381 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.609263 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 08 18:16:43.639480 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.639445 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2df5c42a-be88-430d-9d73-ab199a589304-metrics-client-ca\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.639663 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.639491 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2df5c42a-be88-430d-9d73-ab199a589304-node-exporter-wtmp\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.639663 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.639518 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2df5c42a-be88-430d-9d73-ab199a589304-node-exporter-tls\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.639663 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.639595 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2df5c42a-be88-430d-9d73-ab199a589304-root\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.639663 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.639630 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2df5c42a-be88-430d-9d73-ab199a589304-node-exporter-textfile\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.639812 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.639671 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2df5c42a-be88-430d-9d73-ab199a589304-sys\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.639812 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.639693 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2df5c42a-be88-430d-9d73-ab199a589304-node-exporter-accelerators-collector-config\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.639812 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.639780 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2df5c42a-be88-430d-9d73-ab199a589304-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.639902 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.639815 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtj2s\" (UniqueName: \"kubernetes.io/projected/2df5c42a-be88-430d-9d73-ab199a589304-kube-api-access-xtj2s\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.740265 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.740223 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtj2s\" (UniqueName: \"kubernetes.io/projected/2df5c42a-be88-430d-9d73-ab199a589304-kube-api-access-xtj2s\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.740265 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.740264 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2df5c42a-be88-430d-9d73-ab199a589304-metrics-client-ca\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.740519 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.740291 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2df5c42a-be88-430d-9d73-ab199a589304-node-exporter-wtmp\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.740519 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.740313 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2df5c42a-be88-430d-9d73-ab199a589304-node-exporter-tls\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.740519 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.740334 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2df5c42a-be88-430d-9d73-ab199a589304-root\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.740519 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.740400 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2df5c42a-be88-430d-9d73-ab199a589304-root\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.740519 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.740397 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2df5c42a-be88-430d-9d73-ab199a589304-node-exporter-textfile\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.740519 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.740466 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2df5c42a-be88-430d-9d73-ab199a589304-sys\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.740519 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.740471 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2df5c42a-be88-430d-9d73-ab199a589304-node-exporter-wtmp\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.740519 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.740495 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2df5c42a-be88-430d-9d73-ab199a589304-node-exporter-accelerators-collector-config\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.740519 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.740514 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2df5c42a-be88-430d-9d73-ab199a589304-sys\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.740906 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.740568 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2df5c42a-be88-430d-9d73-ab199a589304-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.740906 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.740769 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2df5c42a-be88-430d-9d73-ab199a589304-node-exporter-textfile\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.741124 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.741104 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2df5c42a-be88-430d-9d73-ab199a589304-node-exporter-accelerators-collector-config\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.741235 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.741216 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2df5c42a-be88-430d-9d73-ab199a589304-metrics-client-ca\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.742824 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.742798 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2df5c42a-be88-430d-9d73-ab199a589304-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.743219 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.743202 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2df5c42a-be88-430d-9d73-ab199a589304-node-exporter-tls\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.747337 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.747314 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtj2s\" (UniqueName: \"kubernetes.io/projected/2df5c42a-be88-430d-9d73-ab199a589304-kube-api-access-xtj2s\") pod \"node-exporter-l8kvq\" (UID: \"2df5c42a-be88-430d-9d73-ab199a589304\") " pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.915138 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:43.915109 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l8kvq" Jun 08 18:16:43.926230 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:16:43.926198 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2df5c42a_be88_430d_9d73_ab199a589304.slice/crio-df38a520ee538d67601a03d4b5688351c43d4d565cdf9c682a50cd13896d45f0 WatchSource:0}: Error finding container df38a520ee538d67601a03d4b5688351c43d4d565cdf9c682a50cd13896d45f0: Status 404 returned error can't find the container with id df38a520ee538d67601a03d4b5688351c43d4d565cdf9c682a50cd13896d45f0 Jun 08 18:16:44.341129 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:44.341039 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l8kvq" event={"ID":"2df5c42a-be88-430d-9d73-ab199a589304","Type":"ContainerStarted","Data":"df38a520ee538d67601a03d4b5688351c43d4d565cdf9c682a50cd13896d45f0"} Jun 08 18:16:45.346146 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:45.346108 2572 generic.go:356] "Generic (PLEG): container finished" podID="2df5c42a-be88-430d-9d73-ab199a589304" containerID="c845217a3c6b5a698d7a953a6eae27198d3ea65c2d09ad84d3a6a545363fdf01" exitCode=0 Jun 08 18:16:45.346531 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:45.346170 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l8kvq" event={"ID":"2df5c42a-be88-430d-9d73-ab199a589304","Type":"ContainerDied","Data":"c845217a3c6b5a698d7a953a6eae27198d3ea65c2d09ad84d3a6a545363fdf01"} Jun 08 18:16:46.351198 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:46.351158 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l8kvq" event={"ID":"2df5c42a-be88-430d-9d73-ab199a589304","Type":"ContainerStarted","Data":"200338e629b9192dd85de2ad52dd316b3880d18a80ad3c6fc72a21482ad5b2c4"} Jun 08 18:16:46.351568 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:46.351218 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l8kvq" event={"ID":"2df5c42a-be88-430d-9d73-ab199a589304","Type":"ContainerStarted","Data":"3a27f91087990936f3916d133f596d3090c9a289108c961726ddbe88951befd6"} Jun 08 18:16:46.370414 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:46.370357 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-l8kvq" podStartSLOduration=2.111626987 podStartE2EDuration="3.37033866s" totalImagesPullingTime="1.193976394s" totalInitContainerRuntime="64.735279ms" isStatefulPod=true podCreationTimestamp="2026-06-08 18:16:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:16:46.369117448 +0000 UTC m=+240.223624308" watchObservedRunningTime="2026-06-08 18:16:46.37033866 +0000 UTC m=+240.224845459" Jun 08 18:16:49.853855 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.853818 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 18:16:49.857354 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.857337 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.859424 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.859401 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 08 18:16:49.859525 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.859470 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 08 18:16:49.859597 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.859574 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6qur38cbugthe\"" Jun 08 18:16:49.859681 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.859662 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 08 18:16:49.859995 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.859980 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 08 18:16:49.860166 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.860144 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-859zp\"" Jun 08 18:16:49.860235 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.860186 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 08 18:16:49.860285 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.860268 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 08 18:16:49.860330 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.860295 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 08 18:16:49.860330 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.860321 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 08 18:16:49.860330 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.860322 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 08 18:16:49.861559 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.861543 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 08 18:16:49.861639 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.861623 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 08 18:16:49.861733 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.861716 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 08 18:16:49.861832 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.861794 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 08 18:16:49.864021 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.864004 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 08 18:16:49.865788 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.865773 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 08 18:16:49.876667 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.876647 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 18:16:49.890371 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.890348 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7d3bea65-1ded-49c4-a382-283bf76cda67-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.890449 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.890372 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7d3bea65-1ded-49c4-a382-283bf76cda67-config-out\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.890449 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.890389 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.890449 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.890405 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.890449 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.890423 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.890601 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.890458 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.890601 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.890534 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.890601 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.890564 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.890601 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.890599 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.890727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.890618 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.890727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.890635 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.890727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.890715 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-web-config\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.890833 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.890756 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-config\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.890833 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.890775 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.890833 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.890806 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.890931 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.890834 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-588xg\" (UniqueName: \"kubernetes.io/projected/7d3bea65-1ded-49c4-a382-283bf76cda67-kube-api-access-588xg\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.890931 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.890870 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.890931 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.890888 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.890931 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.890902 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.890931 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.890920 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.991314 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.991281 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.991314 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.991316 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.991515 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.991340 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.991550 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.991519 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.991608 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.991584 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-web-config\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.991657 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.991623 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-config\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.991657 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.991643 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.991785 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.991748 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.991913 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.991894 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-588xg\" (UniqueName: \"kubernetes.io/projected/7d3bea65-1ded-49c4-a382-283bf76cda67-kube-api-access-588xg\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.992109 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.992090 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.992266 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.992241 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.992266 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.991996 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.992455 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.992278 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.992455 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.992310 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.992455 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.992352 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7d3bea65-1ded-49c4-a382-283bf76cda67-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.992455 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.992375 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7d3bea65-1ded-49c4-a382-283bf76cda67-config-out\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.992455 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.992400 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.992455 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.992428 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.992743 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.992463 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.992743 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.992491 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.992743 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.992524 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.992743 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.991932 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.993299 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.993270 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.995577 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.995548 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.995677 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.995579 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.996110 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.995888 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.996110 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.996005 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.996110 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.996094 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7d3bea65-1ded-49c4-a382-283bf76cda67-config-out\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.997194 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.996438 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.997194 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.996481 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.997194 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.996600 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-web-config\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.997194 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.996801 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7d3bea65-1ded-49c4-a382-283bf76cda67-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.997194 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.996905 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.997194 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.996951 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.997194 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.997006 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.997194 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.997119 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-config\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.997614 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.997200 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.998166 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.998146 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:49.998243 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:49.998226 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:50.001774 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:50.001755 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-588xg\" (UniqueName: \"kubernetes.io/projected/7d3bea65-1ded-49c4-a382-283bf76cda67-kube-api-access-588xg\") pod \"prometheus-k8s-0\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:50.167347 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:50.167308 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:16:50.316721 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:50.316688 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 18:16:50.320119 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:16:50.320089 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d3bea65_1ded_49c4_a382_283bf76cda67.slice/crio-fb8c938563f34b8ec980756844475a56aa9c5059a6118019804fdb95ca113e3c WatchSource:0}: Error finding container fb8c938563f34b8ec980756844475a56aa9c5059a6118019804fdb95ca113e3c: Status 404 returned error can't find the container with id fb8c938563f34b8ec980756844475a56aa9c5059a6118019804fdb95ca113e3c Jun 08 18:16:50.366955 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:50.366922 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d3bea65-1ded-49c4-a382-283bf76cda67","Type":"ContainerStarted","Data":"fb8c938563f34b8ec980756844475a56aa9c5059a6118019804fdb95ca113e3c"} Jun 08 18:16:51.372039 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:51.372008 2572 generic.go:356] "Generic (PLEG): container finished" podID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerID="b025501f51d4056858c744e8934e7aa6e9921f9522ec6d8d81da3a79feaf0d55" exitCode=0 Jun 08 18:16:51.372396 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:51.372102 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d3bea65-1ded-49c4-a382-283bf76cda67","Type":"ContainerDied","Data":"b025501f51d4056858c744e8934e7aa6e9921f9522ec6d8d81da3a79feaf0d55"} Jun 08 18:16:52.247268 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:52.247237 2572 patch_prober.go:28] interesting pod/image-registry-84f4b4799d-455f8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 18:16:52.247427 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:52.247287 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-84f4b4799d-455f8" podUID="79eae03b-1bf3-40dd-adfd-be1fd34224ae" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 18:16:52.618815 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:52.618734 2572 patch_prober.go:28] interesting pod/image-registry-dc69d89dc-lwb8p container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 18:16:52.618815 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:52.618783 2572 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" podUID="ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 18:16:54.311621 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:54.311584 2572 patch_prober.go:28] interesting pod/image-registry-dc69d89dc-lwb8p container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 18:16:54.312061 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:54.311643 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" podUID="ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 18:16:55.389920 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:55.389883 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d3bea65-1ded-49c4-a382-283bf76cda67","Type":"ContainerStarted","Data":"2389f2a043dc1d40c204287fc0962c246ac61691b89bd34fe301ee23b7618a09"} Jun 08 18:16:55.390409 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:55.390017 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d3bea65-1ded-49c4-a382-283bf76cda67","Type":"ContainerStarted","Data":"0a347bcd095a86048ff0b1bef70d4f01994a549dde96403ca9cbeddff5fab09a"} Jun 08 18:16:57.261976 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.261907 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-84f4b4799d-455f8" podUID="79eae03b-1bf3-40dd-adfd-be1fd34224ae" containerName="registry" containerID="cri-o://a3c78e9036ef7d96a0532ce18fc1d5ce736f5b5d80c44532eeaee25a51a1ad74" gracePeriod=30 Jun 08 18:16:57.399775 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.399738 2572 generic.go:356] "Generic (PLEG): container finished" podID="79eae03b-1bf3-40dd-adfd-be1fd34224ae" containerID="a3c78e9036ef7d96a0532ce18fc1d5ce736f5b5d80c44532eeaee25a51a1ad74" exitCode=0 Jun 08 18:16:57.399904 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.399812 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84f4b4799d-455f8" event={"ID":"79eae03b-1bf3-40dd-adfd-be1fd34224ae","Type":"ContainerDied","Data":"a3c78e9036ef7d96a0532ce18fc1d5ce736f5b5d80c44532eeaee25a51a1ad74"} Jun 08 18:16:57.403222 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.403190 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d3bea65-1ded-49c4-a382-283bf76cda67","Type":"ContainerStarted","Data":"5109101dffddd065711386dcfda52cc6ef719281838236f7f992045ba8229cb1"} Jun 08 18:16:57.403327 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.403264 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d3bea65-1ded-49c4-a382-283bf76cda67","Type":"ContainerStarted","Data":"3d614329c08aa1d9615b254bc01d4df14c298c1245c8f0b45cceb9e6bd46dff9"} Jun 08 18:16:57.403327 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.403310 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d3bea65-1ded-49c4-a382-283bf76cda67","Type":"ContainerStarted","Data":"2da001b08c6b67adebfbd4e9232cae9099f372e65b2d859abadd2b17e9438cfb"} Jun 08 18:16:57.403398 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.403353 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d3bea65-1ded-49c4-a382-283bf76cda67","Type":"ContainerStarted","Data":"7229513e924ab9e633ae560a575677705e6232598364bd058a28d2e8c8dbf2fe"} Jun 08 18:16:57.437698 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.437648 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.479718849 podStartE2EDuration="8.437632704s" totalImagesPullingTime="5.940170323s" totalInitContainerRuntime="17.743532ms" isStatefulPod=false podCreationTimestamp="2026-06-08 18:16:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:16:57.43704809 +0000 UTC m=+251.291554888" watchObservedRunningTime="2026-06-08 18:16:57.437632704 +0000 UTC m=+251.292139541" Jun 08 18:16:57.501979 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.501934 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:57.561207 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.561126 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/79eae03b-1bf3-40dd-adfd-be1fd34224ae-image-registry-private-configuration\") pod \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " Jun 08 18:16:57.561207 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.561166 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/79eae03b-1bf3-40dd-adfd-be1fd34224ae-ca-trust-extracted\") pod \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " Jun 08 18:16:57.561207 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.561198 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-bound-sa-token\") pod \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " Jun 08 18:16:57.561471 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.561228 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-tls\") pod \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " Jun 08 18:16:57.561471 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.561249 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjknf\" (UniqueName: \"kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-kube-api-access-rjknf\") pod \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " Jun 08 18:16:57.561471 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.561286 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/79eae03b-1bf3-40dd-adfd-be1fd34224ae-installation-pull-secrets\") pod \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " Jun 08 18:16:57.561471 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.561308 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/79eae03b-1bf3-40dd-adfd-be1fd34224ae-trusted-ca\") pod \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " Jun 08 18:16:57.561471 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.561334 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-certificates\") pod \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\" (UID: \"79eae03b-1bf3-40dd-adfd-be1fd34224ae\") " Jun 08 18:16:57.562049 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.561946 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "79eae03b-1bf3-40dd-adfd-be1fd34224ae" (UID: "79eae03b-1bf3-40dd-adfd-be1fd34224ae"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 18:16:57.562049 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.562007 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79eae03b-1bf3-40dd-adfd-be1fd34224ae-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "79eae03b-1bf3-40dd-adfd-be1fd34224ae" (UID: "79eae03b-1bf3-40dd-adfd-be1fd34224ae"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 18:16:57.563842 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.563778 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-kube-api-access-rjknf" (OuterVolumeSpecName: "kube-api-access-rjknf") pod "79eae03b-1bf3-40dd-adfd-be1fd34224ae" (UID: "79eae03b-1bf3-40dd-adfd-be1fd34224ae"). InnerVolumeSpecName "kube-api-access-rjknf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:16:57.563951 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.563910 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "79eae03b-1bf3-40dd-adfd-be1fd34224ae" (UID: "79eae03b-1bf3-40dd-adfd-be1fd34224ae"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:16:57.563951 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.563913 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79eae03b-1bf3-40dd-adfd-be1fd34224ae-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "79eae03b-1bf3-40dd-adfd-be1fd34224ae" (UID: "79eae03b-1bf3-40dd-adfd-be1fd34224ae"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:16:57.564066 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.563945 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79eae03b-1bf3-40dd-adfd-be1fd34224ae-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "79eae03b-1bf3-40dd-adfd-be1fd34224ae" (UID: "79eae03b-1bf3-40dd-adfd-be1fd34224ae"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:16:57.564126 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.564109 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "79eae03b-1bf3-40dd-adfd-be1fd34224ae" (UID: "79eae03b-1bf3-40dd-adfd-be1fd34224ae"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:16:57.571754 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.571727 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79eae03b-1bf3-40dd-adfd-be1fd34224ae-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "79eae03b-1bf3-40dd-adfd-be1fd34224ae" (UID: "79eae03b-1bf3-40dd-adfd-be1fd34224ae"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 18:16:57.662849 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.662815 2572 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/79eae03b-1bf3-40dd-adfd-be1fd34224ae-image-registry-private-configuration\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:16:57.662849 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.662843 2572 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/79eae03b-1bf3-40dd-adfd-be1fd34224ae-ca-trust-extracted\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:16:57.662849 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.662853 2572 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-bound-sa-token\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:16:57.663233 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.662865 2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-tls\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:16:57.663233 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.662875 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjknf\" (UniqueName: \"kubernetes.io/projected/79eae03b-1bf3-40dd-adfd-be1fd34224ae-kube-api-access-rjknf\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:16:57.663233 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.662884 2572 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/79eae03b-1bf3-40dd-adfd-be1fd34224ae-installation-pull-secrets\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:16:57.663233 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.662892 2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/79eae03b-1bf3-40dd-adfd-be1fd34224ae-trusted-ca\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:16:57.663233 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:57.662900 2572 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/79eae03b-1bf3-40dd-adfd-be1fd34224ae-registry-certificates\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:16:58.408265 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:58.408236 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84f4b4799d-455f8" Jun 08 18:16:58.408627 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:58.408283 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84f4b4799d-455f8" event={"ID":"79eae03b-1bf3-40dd-adfd-be1fd34224ae","Type":"ContainerDied","Data":"fa1e775603938d9c8b2fbcd11e88b7d5ceb1ffee7638a197607549183386fcac"} Jun 08 18:16:58.413814 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:58.409317 2572 scope.go:117] "RemoveContainer" containerID="a3c78e9036ef7d96a0532ce18fc1d5ce736f5b5d80c44532eeaee25a51a1ad74" Jun 08 18:16:58.431455 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:58.431430 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84f4b4799d-455f8"] Jun 08 18:16:58.435079 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:58.435053 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-84f4b4799d-455f8"] Jun 08 18:16:58.470368 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:58.470333 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs\") pod \"network-metrics-daemon-jjs4l\" (UID: \"086390f2-8eb2-4903-a324-bf54451c435a\") " pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:16:58.472665 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:58.472635 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/086390f2-8eb2-4903-a324-bf54451c435a-metrics-certs\") pod \"network-metrics-daemon-jjs4l\" (UID: \"086390f2-8eb2-4903-a324-bf54451c435a\") " pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:16:58.591300 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:58.591265 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79eae03b-1bf3-40dd-adfd-be1fd34224ae" path="/var/lib/kubelet/pods/79eae03b-1bf3-40dd-adfd-be1fd34224ae/volumes" Jun 08 18:16:58.689800 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:58.689713 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-58l4q\"" Jun 08 18:16:58.698364 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:58.698343 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jjs4l" Jun 08 18:16:58.827918 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:58.827886 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jjs4l"] Jun 08 18:16:58.831423 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:16:58.831397 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod086390f2_8eb2_4903_a324_bf54451c435a.slice/crio-50040a670c4bb19912b7f0c6c268cec7d6becf91e3d19a1eda33d260adc8f1bb WatchSource:0}: Error finding container 50040a670c4bb19912b7f0c6c268cec7d6becf91e3d19a1eda33d260adc8f1bb: Status 404 returned error can't find the container with id 50040a670c4bb19912b7f0c6c268cec7d6becf91e3d19a1eda33d260adc8f1bb Jun 08 18:16:59.415043 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:16:59.415008 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jjs4l" event={"ID":"086390f2-8eb2-4903-a324-bf54451c435a","Type":"ContainerStarted","Data":"50040a670c4bb19912b7f0c6c268cec7d6becf91e3d19a1eda33d260adc8f1bb"} Jun 08 18:17:00.168442 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:00.168409 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:17:00.420411 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:00.420318 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jjs4l" event={"ID":"086390f2-8eb2-4903-a324-bf54451c435a","Type":"ContainerStarted","Data":"8ff237df73681a5e3c3eabe925d74c4c32c9493b3a3db660bd09ecd16b85fe3e"} Jun 08 18:17:00.420411 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:00.420380 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jjs4l" event={"ID":"086390f2-8eb2-4903-a324-bf54451c435a","Type":"ContainerStarted","Data":"2444286a4450bec595fac6534f40f26813076dce06898faf986d95e83782e710"} Jun 08 18:17:00.447269 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:00.447222 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jjs4l" podStartSLOduration=253.56023226 podStartE2EDuration="4m14.447206553s" totalImagesPullingTime="886.974293ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:17:00.446329711 +0000 UTC m=+254.300836520" watchObservedRunningTime="2026-06-08 18:17:00.447206553 +0000 UTC m=+254.301713348" Jun 08 18:17:02.618671 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:02.618639 2572 patch_prober.go:28] interesting pod/image-registry-dc69d89dc-lwb8p container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 18:17:02.619080 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:02.618688 2572 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" podUID="ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 18:17:04.312046 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:04.311942 2572 patch_prober.go:28] interesting pod/image-registry-dc69d89dc-lwb8p container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 18:17:04.312046 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:04.312023 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" podUID="ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 18:17:12.618307 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:12.618272 2572 patch_prober.go:28] interesting pod/image-registry-dc69d89dc-lwb8p container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 18:17:12.618726 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:12.618334 2572 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" podUID="ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 18:17:12.618726 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:12.618370 2572 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:17:12.618857 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:12.618777 2572 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"e4e00b9ec541186714d38b6f4e7c0737a1e22b5e5a519894b0292c2c50344f06"} pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" containerMessage="Container registry failed liveness probe, will be restarted" Jun 08 18:17:12.622358 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:12.622322 2572 patch_prober.go:28] interesting pod/image-registry-dc69d89dc-lwb8p container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 18:17:12.622467 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:12.622374 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" podUID="ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 18:17:22.622765 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:22.622722 2572 patch_prober.go:28] interesting pod/image-registry-dc69d89dc-lwb8p container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 18:17:22.623153 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:22.622803 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" podUID="ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 18:17:24.513152 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:24.513117 2572 generic.go:356] "Generic (PLEG): container finished" podID="62df564d-0c6d-4fd1-a67c-de8d9dc9d5a7" containerID="41df78ea43fcefe7c6facd234665c65ea3a52d1677cd29f0a58b346dd76ef119" exitCode=0 Jun 08 18:17:24.513528 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:24.513161 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-fl7lq" event={"ID":"62df564d-0c6d-4fd1-a67c-de8d9dc9d5a7","Type":"ContainerDied","Data":"41df78ea43fcefe7c6facd234665c65ea3a52d1677cd29f0a58b346dd76ef119"} Jun 08 18:17:24.513528 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:24.513459 2572 scope.go:117] "RemoveContainer" containerID="41df78ea43fcefe7c6facd234665c65ea3a52d1677cd29f0a58b346dd76ef119" Jun 08 18:17:25.517982 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:25.517933 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-fl7lq" event={"ID":"62df564d-0c6d-4fd1-a67c-de8d9dc9d5a7","Type":"ContainerStarted","Data":"92803b1608999969648b02b5bbe83f918095ecaf3154b1fcdf8a3bc355f70c4b"} Jun 08 18:17:26.522932 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:26.522846 2572 generic.go:356] "Generic (PLEG): container finished" podID="7fcdb8ae-ba39-465c-9112-3f39e2e9032a" containerID="23cafd382ec1eb3680663110f9c8e5e8e08ae9801b5a1433924c481784dcce0c" exitCode=255 Jun 08 18:17:26.522932 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:26.522922 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65584f7cd6-whmlg" event={"ID":"7fcdb8ae-ba39-465c-9112-3f39e2e9032a","Type":"ContainerDied","Data":"23cafd382ec1eb3680663110f9c8e5e8e08ae9801b5a1433924c481784dcce0c"} Jun 08 18:17:26.523433 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:26.522990 2572 scope.go:117] "RemoveContainer" containerID="19c7d7dfda0ae963924f283752a1de51a965c9284302eca298dad99fd101dc9a" Jun 08 18:17:26.529388 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:26.529367 2572 scope.go:117] "RemoveContainer" containerID="23cafd382ec1eb3680663110f9c8e5e8e08ae9801b5a1433924c481784dcce0c" Jun 08 18:17:26.529610 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:17:26.529579 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-65584f7cd6-whmlg_open-cluster-management-agent-addon(7fcdb8ae-ba39-465c-9112-3f39e2e9032a)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65584f7cd6-whmlg" podUID="7fcdb8ae-ba39-465c-9112-3f39e2e9032a" Jun 08 18:17:27.084468 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:17:27.084410 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-n4m82" podUID="c2665ffe-cccc-432a-85a7-a5244f0b2bc8" Jun 08 18:17:27.527324 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:27.527294 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n4m82" Jun 08 18:17:30.933385 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:30.933342 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls\") pod \"dns-default-n4m82\" (UID: \"c2665ffe-cccc-432a-85a7-a5244f0b2bc8\") " pod="openshift-dns/dns-default-n4m82" Jun 08 18:17:30.935654 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:30.935631 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2665ffe-cccc-432a-85a7-a5244f0b2bc8-metrics-tls\") pod \"dns-default-n4m82\" (UID: \"c2665ffe-cccc-432a-85a7-a5244f0b2bc8\") " pod="openshift-dns/dns-default-n4m82" Jun 08 18:17:31.130574 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:31.130541 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s24pr\"" Jun 08 18:17:31.138981 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:31.138936 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n4m82" Jun 08 18:17:31.269718 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:31.269681 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n4m82"] Jun 08 18:17:31.273020 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:17:31.272984 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2665ffe_cccc_432a_85a7_a5244f0b2bc8.slice/crio-dd3017d91caebe4e15bb2f986859ff092edd76cf5831867f241a2fec0b715575 WatchSource:0}: Error finding container dd3017d91caebe4e15bb2f986859ff092edd76cf5831867f241a2fec0b715575: Status 404 returned error can't find the container with id dd3017d91caebe4e15bb2f986859ff092edd76cf5831867f241a2fec0b715575 Jun 08 18:17:31.546273 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:31.546193 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n4m82" event={"ID":"c2665ffe-cccc-432a-85a7-a5244f0b2bc8","Type":"ContainerStarted","Data":"dd3017d91caebe4e15bb2f986859ff092edd76cf5831867f241a2fec0b715575"} Jun 08 18:17:32.377137 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:32.377097 2572 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65584f7cd6-whmlg" Jun 08 18:17:32.377594 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:32.377557 2572 scope.go:117] "RemoveContainer" containerID="23cafd382ec1eb3680663110f9c8e5e8e08ae9801b5a1433924c481784dcce0c" Jun 08 18:17:32.377776 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:17:32.377745 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-65584f7cd6-whmlg_open-cluster-management-agent-addon(7fcdb8ae-ba39-465c-9112-3f39e2e9032a)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65584f7cd6-whmlg" podUID="7fcdb8ae-ba39-465c-9112-3f39e2e9032a" Jun 08 18:17:32.624169 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:32.624132 2572 patch_prober.go:28] interesting pod/image-registry-dc69d89dc-lwb8p container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 18:17:32.624383 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:32.624195 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" podUID="ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 18:17:33.555117 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:33.555082 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n4m82" event={"ID":"c2665ffe-cccc-432a-85a7-a5244f0b2bc8","Type":"ContainerStarted","Data":"54d7489683ead344b64dcf6c5473b7dc6a8a18e4c846807c661920db38eb6a3e"} Jun 08 18:17:33.555571 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:33.555143 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n4m82" event={"ID":"c2665ffe-cccc-432a-85a7-a5244f0b2bc8","Type":"ContainerStarted","Data":"488cbc46ae71c63ff0071bb601921aca038932b6aabf247f03b5d8c82137481c"} Jun 08 18:17:33.555571 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:33.555232 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-n4m82" Jun 08 18:17:33.573448 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:33.573395 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-n4m82" podStartSLOduration=251.981803532 podStartE2EDuration="4m13.573378795s" totalImagesPullingTime="1.591575263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:13:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:17:33.571714969 +0000 UTC m=+287.426221757" watchObservedRunningTime="2026-06-08 18:17:33.573378795 +0000 UTC m=+287.427885593" Jun 08 18:17:37.637468 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:37.637423 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" podUID="ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b" containerName="registry" containerID="cri-o://e4e00b9ec541186714d38b6f4e7c0737a1e22b5e5a519894b0292c2c50344f06" gracePeriod=30 Jun 08 18:17:39.577212 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:39.577181 2572 generic.go:356] "Generic (PLEG): container finished" podID="ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b" containerID="e4e00b9ec541186714d38b6f4e7c0737a1e22b5e5a519894b0292c2c50344f06" exitCode=0 Jun 08 18:17:39.577631 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:39.577257 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" event={"ID":"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b","Type":"ContainerDied","Data":"e4e00b9ec541186714d38b6f4e7c0737a1e22b5e5a519894b0292c2c50344f06"} Jun 08 18:17:39.577631 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:39.577314 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" event={"ID":"ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b","Type":"ContainerStarted","Data":"207f8f5c3329baf0ae7fdad676ff99a0bd6726c99805d815cf2775cceb1fd886"} Jun 08 18:17:39.577631 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:39.577479 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:17:42.141557 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:42.141528 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l8kvq_2df5c42a-be88-430d-9d73-ab199a589304/init-textfile/0.log" Jun 08 18:17:42.320120 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:42.320092 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l8kvq_2df5c42a-be88-430d-9d73-ab199a589304/node-exporter/0.log" Jun 08 18:17:42.518396 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:42.518361 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l8kvq_2df5c42a-be88-430d-9d73-ab199a589304/kube-rbac-proxy/0.log" Jun 08 18:17:43.319015 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:43.318990 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7d3bea65-1ded-49c4-a382-283bf76cda67/init-config-reloader/0.log" Jun 08 18:17:43.518938 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:43.518908 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7d3bea65-1ded-49c4-a382-283bf76cda67/prometheus/0.log" Jun 08 18:17:43.561987 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:43.561931 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-n4m82" Jun 08 18:17:43.586860 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:43.586779 2572 scope.go:117] "RemoveContainer" containerID="23cafd382ec1eb3680663110f9c8e5e8e08ae9801b5a1433924c481784dcce0c" Jun 08 18:17:43.718782 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:43.718748 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7d3bea65-1ded-49c4-a382-283bf76cda67/config-reloader/0.log" Jun 08 18:17:43.919136 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:43.919110 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7d3bea65-1ded-49c4-a382-283bf76cda67/thanos-sidecar/0.log" Jun 08 18:17:44.118477 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:44.118451 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7d3bea65-1ded-49c4-a382-283bf76cda67/kube-rbac-proxy-web/0.log" Jun 08 18:17:44.319761 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:44.319675 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7d3bea65-1ded-49c4-a382-283bf76cda67/kube-rbac-proxy/0.log" Jun 08 18:17:44.517918 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:44.517892 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7d3bea65-1ded-49c4-a382-283bf76cda67/kube-rbac-proxy-thanos/0.log" Jun 08 18:17:44.605455 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:44.605371 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65584f7cd6-whmlg" event={"ID":"7fcdb8ae-ba39-465c-9112-3f39e2e9032a","Type":"ContainerStarted","Data":"8ea99bc6fb6401eea5cd7eb1d6296277dcc3060c000456635b1ba94ea9ccc272"} Jun 08 18:17:46.543123 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:46.543100 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 18:17:46.543492 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:46.543354 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 18:17:46.546476 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:46.546458 2572 kubelet.go:1648] "Image garbage collection succeeded" Jun 08 18:17:48.520064 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:48.520034 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n4m82_c2665ffe-cccc-432a-85a7-a5244f0b2bc8/dns/0.log" Jun 08 18:17:48.720876 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:48.720841 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n4m82_c2665ffe-cccc-432a-85a7-a5244f0b2bc8/kube-rbac-proxy/0.log" Jun 08 18:17:49.319345 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:49.319319 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z77dq_1ad02886-8d99-4ad6-8103-a3eafea4af48/dns-node-resolver/0.log" Jun 08 18:17:49.718870 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:49.718836 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-dc69d89dc-lwb8p_ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b/registry/1.log" Jun 08 18:17:50.119301 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:50.119227 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bvg6f_445f7d76-73b2-4d8c-aad9-9195be1bd5fa/node-ca/0.log" Jun 08 18:17:50.167605 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:50.167570 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:17:50.183318 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:50.183294 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:17:50.520204 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:50.520168 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d8bbf5d9c-f27lp_685e569f-65a8-4c9b-b402-dc7ce33102ff/router/0.log" Jun 08 18:17:50.647251 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:50.647223 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:17:50.730502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:17:50.730466 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bl784_0a6fe03e-3966-431d-99ff-43a27c4d0249/serve-healthcheck-canary/0.log" Jun 08 18:18:00.585501 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:00.585468 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-dc69d89dc-lwb8p" Jun 08 18:18:08.296928 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:08.296890 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 18:18:08.297356 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:08.297177 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="prometheus" containerID="cri-o://0a347bcd095a86048ff0b1bef70d4f01994a549dde96403ca9cbeddff5fab09a" gracePeriod=600 Jun 08 18:18:08.297356 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:08.297210 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="kube-rbac-proxy" containerID="cri-o://3d614329c08aa1d9615b254bc01d4df14c298c1245c8f0b45cceb9e6bd46dff9" gracePeriod=600 Jun 08 18:18:08.297356 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:08.297219 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="thanos-sidecar" containerID="cri-o://7229513e924ab9e633ae560a575677705e6232598364bd058a28d2e8c8dbf2fe" gracePeriod=600 Jun 08 18:18:08.297356 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:08.297237 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="kube-rbac-proxy-web" containerID="cri-o://2da001b08c6b67adebfbd4e9232cae9099f372e65b2d859abadd2b17e9438cfb" gracePeriod=600 Jun 08 18:18:08.297356 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:08.297287 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="kube-rbac-proxy-thanos" containerID="cri-o://5109101dffddd065711386dcfda52cc6ef719281838236f7f992045ba8229cb1" gracePeriod=600 Jun 08 18:18:08.297356 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:08.297295 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="config-reloader" containerID="cri-o://2389f2a043dc1d40c204287fc0962c246ac61691b89bd34fe301ee23b7618a09" gracePeriod=600 Jun 08 18:18:08.702794 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:08.702760 2572 generic.go:356] "Generic (PLEG): container finished" podID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerID="5109101dffddd065711386dcfda52cc6ef719281838236f7f992045ba8229cb1" exitCode=0 Jun 08 18:18:08.702794 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:08.702787 2572 generic.go:356] "Generic (PLEG): container finished" podID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerID="3d614329c08aa1d9615b254bc01d4df14c298c1245c8f0b45cceb9e6bd46dff9" exitCode=0 Jun 08 18:18:08.702794 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:08.702793 2572 generic.go:356] "Generic (PLEG): container finished" podID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerID="7229513e924ab9e633ae560a575677705e6232598364bd058a28d2e8c8dbf2fe" exitCode=0 Jun 08 18:18:08.702794 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:08.702799 2572 generic.go:356] "Generic (PLEG): container finished" podID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerID="2389f2a043dc1d40c204287fc0962c246ac61691b89bd34fe301ee23b7618a09" exitCode=0 Jun 08 18:18:08.702794 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:08.702804 2572 generic.go:356] "Generic (PLEG): container finished" podID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerID="0a347bcd095a86048ff0b1bef70d4f01994a549dde96403ca9cbeddff5fab09a" exitCode=0 Jun 08 18:18:08.703163 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:08.702842 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d3bea65-1ded-49c4-a382-283bf76cda67","Type":"ContainerDied","Data":"5109101dffddd065711386dcfda52cc6ef719281838236f7f992045ba8229cb1"} Jun 08 18:18:08.703163 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:08.702905 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d3bea65-1ded-49c4-a382-283bf76cda67","Type":"ContainerDied","Data":"3d614329c08aa1d9615b254bc01d4df14c298c1245c8f0b45cceb9e6bd46dff9"} Jun 08 18:18:08.703163 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:08.702931 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d3bea65-1ded-49c4-a382-283bf76cda67","Type":"ContainerDied","Data":"7229513e924ab9e633ae560a575677705e6232598364bd058a28d2e8c8dbf2fe"} Jun 08 18:18:08.703163 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:08.702955 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d3bea65-1ded-49c4-a382-283bf76cda67","Type":"ContainerDied","Data":"2389f2a043dc1d40c204287fc0962c246ac61691b89bd34fe301ee23b7618a09"} Jun 08 18:18:08.703163 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:08.702997 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d3bea65-1ded-49c4-a382-283bf76cda67","Type":"ContainerDied","Data":"0a347bcd095a86048ff0b1bef70d4f01994a549dde96403ca9cbeddff5fab09a"} Jun 08 18:18:09.542892 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.542869 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.642100 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642009 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-trusted-ca-bundle\") pod \"7d3bea65-1ded-49c4-a382-283bf76cda67\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " Jun 08 18:18:09.642100 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642049 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-web-config\") pod \"7d3bea65-1ded-49c4-a382-283bf76cda67\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " Jun 08 18:18:09.642100 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642069 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-588xg\" (UniqueName: \"kubernetes.io/projected/7d3bea65-1ded-49c4-a382-283bf76cda67-kube-api-access-588xg\") pod \"7d3bea65-1ded-49c4-a382-283bf76cda67\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " Jun 08 18:18:09.642100 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642087 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-configmap-metrics-client-ca\") pod \"7d3bea65-1ded-49c4-a382-283bf76cda67\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " Jun 08 18:18:09.642429 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642123 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7d3bea65-1ded-49c4-a382-283bf76cda67-tls-assets\") pod \"7d3bea65-1ded-49c4-a382-283bf76cda67\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " Jun 08 18:18:09.642429 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642149 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-config\") pod \"7d3bea65-1ded-49c4-a382-283bf76cda67\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " Jun 08 18:18:09.642429 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642182 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"7d3bea65-1ded-49c4-a382-283bf76cda67\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " Jun 08 18:18:09.642429 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642209 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-grpc-tls\") pod \"7d3bea65-1ded-49c4-a382-283bf76cda67\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " Jun 08 18:18:09.642429 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642246 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-rulefiles-1\") pod \"7d3bea65-1ded-49c4-a382-283bf76cda67\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " Jun 08 18:18:09.642429 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642270 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-kube-rbac-proxy\") pod \"7d3bea65-1ded-49c4-a382-283bf76cda67\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " Jun 08 18:18:09.642429 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642294 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-prometheus-k8s-tls\") pod \"7d3bea65-1ded-49c4-a382-283bf76cda67\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " Jun 08 18:18:09.642429 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642324 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"7d3bea65-1ded-49c4-a382-283bf76cda67\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " Jun 08 18:18:09.642429 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642371 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-rulefiles-0\") pod \"7d3bea65-1ded-49c4-a382-283bf76cda67\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " Jun 08 18:18:09.642429 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642404 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-configmap-serving-certs-ca-bundle\") pod \"7d3bea65-1ded-49c4-a382-283bf76cda67\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " Jun 08 18:18:09.642429 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642429 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-db\") pod \"7d3bea65-1ded-49c4-a382-283bf76cda67\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " Jun 08 18:18:09.643253 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642467 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-configmap-kubelet-serving-ca-bundle\") pod \"7d3bea65-1ded-49c4-a382-283bf76cda67\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " Jun 08 18:18:09.643253 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642484 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "7d3bea65-1ded-49c4-a382-283bf76cda67" (UID: "7d3bea65-1ded-49c4-a382-283bf76cda67"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 18:18:09.643253 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642496 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-metrics-client-certs\") pod \"7d3bea65-1ded-49c4-a382-283bf76cda67\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " Jun 08 18:18:09.643253 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642521 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "7d3bea65-1ded-49c4-a382-283bf76cda67" (UID: "7d3bea65-1ded-49c4-a382-283bf76cda67"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 18:18:09.643253 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642556 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7d3bea65-1ded-49c4-a382-283bf76cda67-config-out\") pod \"7d3bea65-1ded-49c4-a382-283bf76cda67\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " Jun 08 18:18:09.643253 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642588 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-thanos-prometheus-http-client-file\") pod \"7d3bea65-1ded-49c4-a382-283bf76cda67\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " Jun 08 18:18:09.643253 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642621 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-rulefiles-2\") pod \"7d3bea65-1ded-49c4-a382-283bf76cda67\" (UID: \"7d3bea65-1ded-49c4-a382-283bf76cda67\") " Jun 08 18:18:09.643253 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642893 2572 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:18:09.643253 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.642914 2572 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-configmap-metrics-client-ca\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:18:09.643690 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.643346 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "7d3bea65-1ded-49c4-a382-283bf76cda67" (UID: "7d3bea65-1ded-49c4-a382-283bf76cda67"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 18:18:09.644350 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.644316 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "7d3bea65-1ded-49c4-a382-283bf76cda67" (UID: "7d3bea65-1ded-49c4-a382-283bf76cda67"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 18:18:09.644690 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.644662 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "7d3bea65-1ded-49c4-a382-283bf76cda67" (UID: "7d3bea65-1ded-49c4-a382-283bf76cda67"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 18:18:09.645090 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.645064 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "7d3bea65-1ded-49c4-a382-283bf76cda67" (UID: "7d3bea65-1ded-49c4-a382-283bf76cda67"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 18:18:09.645379 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.645352 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d3bea65-1ded-49c4-a382-283bf76cda67-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "7d3bea65-1ded-49c4-a382-283bf76cda67" (UID: "7d3bea65-1ded-49c4-a382-283bf76cda67"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:18:09.645458 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.645384 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d3bea65-1ded-49c4-a382-283bf76cda67-kube-api-access-588xg" (OuterVolumeSpecName: "kube-api-access-588xg") pod "7d3bea65-1ded-49c4-a382-283bf76cda67" (UID: "7d3bea65-1ded-49c4-a382-283bf76cda67"). InnerVolumeSpecName "kube-api-access-588xg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:18:09.646014 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.645985 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "7d3bea65-1ded-49c4-a382-283bf76cda67" (UID: "7d3bea65-1ded-49c4-a382-283bf76cda67"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:18:09.646133 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.646097 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-config" (OuterVolumeSpecName: "config") pod "7d3bea65-1ded-49c4-a382-283bf76cda67" (UID: "7d3bea65-1ded-49c4-a382-283bf76cda67"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:18:09.646133 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.646119 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "7d3bea65-1ded-49c4-a382-283bf76cda67" (UID: "7d3bea65-1ded-49c4-a382-283bf76cda67"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 18:18:09.646460 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.646430 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d3bea65-1ded-49c4-a382-283bf76cda67-config-out" (OuterVolumeSpecName: "config-out") pod "7d3bea65-1ded-49c4-a382-283bf76cda67" (UID: "7d3bea65-1ded-49c4-a382-283bf76cda67"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 18:18:09.646532 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.646480 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "7d3bea65-1ded-49c4-a382-283bf76cda67" (UID: "7d3bea65-1ded-49c4-a382-283bf76cda67"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 18:18:09.646595 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.646532 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "7d3bea65-1ded-49c4-a382-283bf76cda67" (UID: "7d3bea65-1ded-49c4-a382-283bf76cda67"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:18:09.646879 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.646844 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "7d3bea65-1ded-49c4-a382-283bf76cda67" (UID: "7d3bea65-1ded-49c4-a382-283bf76cda67"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:18:09.647055 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.647030 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "7d3bea65-1ded-49c4-a382-283bf76cda67" (UID: "7d3bea65-1ded-49c4-a382-283bf76cda67"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:18:09.648316 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.648291 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "7d3bea65-1ded-49c4-a382-283bf76cda67" (UID: "7d3bea65-1ded-49c4-a382-283bf76cda67"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:18:09.648579 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.648558 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "7d3bea65-1ded-49c4-a382-283bf76cda67" (UID: "7d3bea65-1ded-49c4-a382-283bf76cda67"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:18:09.648641 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.648572 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "7d3bea65-1ded-49c4-a382-283bf76cda67" (UID: "7d3bea65-1ded-49c4-a382-283bf76cda67"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:18:09.655173 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.655152 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-web-config" (OuterVolumeSpecName: "web-config") pod "7d3bea65-1ded-49c4-a382-283bf76cda67" (UID: "7d3bea65-1ded-49c4-a382-283bf76cda67"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:18:09.709562 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.709534 2572 generic.go:356] "Generic (PLEG): container finished" podID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerID="2da001b08c6b67adebfbd4e9232cae9099f372e65b2d859abadd2b17e9438cfb" exitCode=0 Jun 08 18:18:09.709658 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.709572 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d3bea65-1ded-49c4-a382-283bf76cda67","Type":"ContainerDied","Data":"2da001b08c6b67adebfbd4e9232cae9099f372e65b2d859abadd2b17e9438cfb"} Jun 08 18:18:09.709658 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.709622 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.709658 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.709644 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7d3bea65-1ded-49c4-a382-283bf76cda67","Type":"ContainerDied","Data":"fb8c938563f34b8ec980756844475a56aa9c5059a6118019804fdb95ca113e3c"} Jun 08 18:18:09.709756 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.709693 2572 scope.go:117] "RemoveContainer" containerID="5109101dffddd065711386dcfda52cc6ef719281838236f7f992045ba8229cb1" Jun 08 18:18:09.717052 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.717033 2572 scope.go:117] "RemoveContainer" containerID="3d614329c08aa1d9615b254bc01d4df14c298c1245c8f0b45cceb9e6bd46dff9" Jun 08 18:18:09.723554 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.723537 2572 scope.go:117] "RemoveContainer" containerID="2da001b08c6b67adebfbd4e9232cae9099f372e65b2d859abadd2b17e9438cfb" Jun 08 18:18:09.729677 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.729660 2572 scope.go:117] "RemoveContainer" containerID="7229513e924ab9e633ae560a575677705e6232598364bd058a28d2e8c8dbf2fe" Jun 08 18:18:09.734093 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.734065 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 18:18:09.736849 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.736834 2572 scope.go:117] "RemoveContainer" containerID="2389f2a043dc1d40c204287fc0962c246ac61691b89bd34fe301ee23b7618a09" Jun 08 18:18:09.739114 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.739096 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 18:18:09.743395 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.743374 2572 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:18:09.743395 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.743397 2572 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-db\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:18:09.743521 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.743408 2572 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:18:09.743521 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.743416 2572 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-metrics-client-certs\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:18:09.743521 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.743425 2572 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7d3bea65-1ded-49c4-a382-283bf76cda67-config-out\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:18:09.743521 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.743433 2572 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:18:09.743521 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.743442 2572 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:18:09.743521 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.743468 2572 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-web-config\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:18:09.743521 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.743477 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-588xg\" (UniqueName: \"kubernetes.io/projected/7d3bea65-1ded-49c4-a382-283bf76cda67-kube-api-access-588xg\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:18:09.743521 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.743485 2572 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7d3bea65-1ded-49c4-a382-283bf76cda67-tls-assets\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:18:09.743521 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.743494 2572 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-config\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:18:09.743521 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.743502 2572 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:18:09.743521 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.743510 2572 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-grpc-tls\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:18:09.743521 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.743519 2572 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:18:09.743933 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.743528 2572 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-kube-rbac-proxy\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:18:09.743933 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.743537 2572 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:18:09.743933 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.743545 2572 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7d3bea65-1ded-49c4-a382-283bf76cda67-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:18:09.743933 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.743554 2572 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7d3bea65-1ded-49c4-a382-283bf76cda67-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:18:09.743933 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.743632 2572 scope.go:117] "RemoveContainer" containerID="0a347bcd095a86048ff0b1bef70d4f01994a549dde96403ca9cbeddff5fab09a" Jun 08 18:18:09.751609 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.751588 2572 scope.go:117] "RemoveContainer" containerID="b025501f51d4056858c744e8934e7aa6e9921f9522ec6d8d81da3a79feaf0d55" Jun 08 18:18:09.758040 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.758022 2572 scope.go:117] "RemoveContainer" containerID="5109101dffddd065711386dcfda52cc6ef719281838236f7f992045ba8229cb1" Jun 08 18:18:09.758292 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:18:09.758268 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5109101dffddd065711386dcfda52cc6ef719281838236f7f992045ba8229cb1\": container with ID starting with 5109101dffddd065711386dcfda52cc6ef719281838236f7f992045ba8229cb1 not found: ID does not exist" containerID="5109101dffddd065711386dcfda52cc6ef719281838236f7f992045ba8229cb1" Jun 08 18:18:09.758355 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.758301 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5109101dffddd065711386dcfda52cc6ef719281838236f7f992045ba8229cb1"} err="failed to get container status \"5109101dffddd065711386dcfda52cc6ef719281838236f7f992045ba8229cb1\": rpc error: code = NotFound desc = could not find container \"5109101dffddd065711386dcfda52cc6ef719281838236f7f992045ba8229cb1\": container with ID starting with 5109101dffddd065711386dcfda52cc6ef719281838236f7f992045ba8229cb1 not found: ID does not exist" Jun 08 18:18:09.758355 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.758318 2572 scope.go:117] "RemoveContainer" containerID="3d614329c08aa1d9615b254bc01d4df14c298c1245c8f0b45cceb9e6bd46dff9" Jun 08 18:18:09.758567 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:18:09.758551 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d614329c08aa1d9615b254bc01d4df14c298c1245c8f0b45cceb9e6bd46dff9\": container with ID starting with 3d614329c08aa1d9615b254bc01d4df14c298c1245c8f0b45cceb9e6bd46dff9 not found: ID does not exist" containerID="3d614329c08aa1d9615b254bc01d4df14c298c1245c8f0b45cceb9e6bd46dff9" Jun 08 18:18:09.758612 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.758569 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d614329c08aa1d9615b254bc01d4df14c298c1245c8f0b45cceb9e6bd46dff9"} err="failed to get container status \"3d614329c08aa1d9615b254bc01d4df14c298c1245c8f0b45cceb9e6bd46dff9\": rpc error: code = NotFound desc = could not find container \"3d614329c08aa1d9615b254bc01d4df14c298c1245c8f0b45cceb9e6bd46dff9\": container with ID starting with 3d614329c08aa1d9615b254bc01d4df14c298c1245c8f0b45cceb9e6bd46dff9 not found: ID does not exist" Jun 08 18:18:09.758612 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.758582 2572 scope.go:117] "RemoveContainer" containerID="2da001b08c6b67adebfbd4e9232cae9099f372e65b2d859abadd2b17e9438cfb" Jun 08 18:18:09.758769 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:18:09.758750 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2da001b08c6b67adebfbd4e9232cae9099f372e65b2d859abadd2b17e9438cfb\": container with ID starting with 2da001b08c6b67adebfbd4e9232cae9099f372e65b2d859abadd2b17e9438cfb not found: ID does not exist" containerID="2da001b08c6b67adebfbd4e9232cae9099f372e65b2d859abadd2b17e9438cfb" Jun 08 18:18:09.758809 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.758772 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2da001b08c6b67adebfbd4e9232cae9099f372e65b2d859abadd2b17e9438cfb"} err="failed to get container status \"2da001b08c6b67adebfbd4e9232cae9099f372e65b2d859abadd2b17e9438cfb\": rpc error: code = NotFound desc = could not find container \"2da001b08c6b67adebfbd4e9232cae9099f372e65b2d859abadd2b17e9438cfb\": container with ID starting with 2da001b08c6b67adebfbd4e9232cae9099f372e65b2d859abadd2b17e9438cfb not found: ID does not exist" Jun 08 18:18:09.758809 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.758784 2572 scope.go:117] "RemoveContainer" containerID="7229513e924ab9e633ae560a575677705e6232598364bd058a28d2e8c8dbf2fe" Jun 08 18:18:09.759024 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:18:09.759005 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7229513e924ab9e633ae560a575677705e6232598364bd058a28d2e8c8dbf2fe\": container with ID starting with 7229513e924ab9e633ae560a575677705e6232598364bd058a28d2e8c8dbf2fe not found: ID does not exist" containerID="7229513e924ab9e633ae560a575677705e6232598364bd058a28d2e8c8dbf2fe" Jun 08 18:18:09.759097 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.759029 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7229513e924ab9e633ae560a575677705e6232598364bd058a28d2e8c8dbf2fe"} err="failed to get container status \"7229513e924ab9e633ae560a575677705e6232598364bd058a28d2e8c8dbf2fe\": rpc error: code = NotFound desc = could not find container \"7229513e924ab9e633ae560a575677705e6232598364bd058a28d2e8c8dbf2fe\": container with ID starting with 7229513e924ab9e633ae560a575677705e6232598364bd058a28d2e8c8dbf2fe not found: ID does not exist" Jun 08 18:18:09.759097 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.759044 2572 scope.go:117] "RemoveContainer" containerID="2389f2a043dc1d40c204287fc0962c246ac61691b89bd34fe301ee23b7618a09" Jun 08 18:18:09.759250 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:18:09.759231 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2389f2a043dc1d40c204287fc0962c246ac61691b89bd34fe301ee23b7618a09\": container with ID starting with 2389f2a043dc1d40c204287fc0962c246ac61691b89bd34fe301ee23b7618a09 not found: ID does not exist" containerID="2389f2a043dc1d40c204287fc0962c246ac61691b89bd34fe301ee23b7618a09" Jun 08 18:18:09.759304 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.759256 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2389f2a043dc1d40c204287fc0962c246ac61691b89bd34fe301ee23b7618a09"} err="failed to get container status \"2389f2a043dc1d40c204287fc0962c246ac61691b89bd34fe301ee23b7618a09\": rpc error: code = NotFound desc = could not find container \"2389f2a043dc1d40c204287fc0962c246ac61691b89bd34fe301ee23b7618a09\": container with ID starting with 2389f2a043dc1d40c204287fc0962c246ac61691b89bd34fe301ee23b7618a09 not found: ID does not exist" Jun 08 18:18:09.759304 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.759272 2572 scope.go:117] "RemoveContainer" containerID="0a347bcd095a86048ff0b1bef70d4f01994a549dde96403ca9cbeddff5fab09a" Jun 08 18:18:09.759487 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:18:09.759472 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a347bcd095a86048ff0b1bef70d4f01994a549dde96403ca9cbeddff5fab09a\": container with ID starting with 0a347bcd095a86048ff0b1bef70d4f01994a549dde96403ca9cbeddff5fab09a not found: ID does not exist" containerID="0a347bcd095a86048ff0b1bef70d4f01994a549dde96403ca9cbeddff5fab09a" Jun 08 18:18:09.759522 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.759492 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a347bcd095a86048ff0b1bef70d4f01994a549dde96403ca9cbeddff5fab09a"} err="failed to get container status \"0a347bcd095a86048ff0b1bef70d4f01994a549dde96403ca9cbeddff5fab09a\": rpc error: code = NotFound desc = could not find container \"0a347bcd095a86048ff0b1bef70d4f01994a549dde96403ca9cbeddff5fab09a\": container with ID starting with 0a347bcd095a86048ff0b1bef70d4f01994a549dde96403ca9cbeddff5fab09a not found: ID does not exist" Jun 08 18:18:09.759522 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.759505 2572 scope.go:117] "RemoveContainer" containerID="b025501f51d4056858c744e8934e7aa6e9921f9522ec6d8d81da3a79feaf0d55" Jun 08 18:18:09.759718 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:18:09.759701 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b025501f51d4056858c744e8934e7aa6e9921f9522ec6d8d81da3a79feaf0d55\": container with ID starting with b025501f51d4056858c744e8934e7aa6e9921f9522ec6d8d81da3a79feaf0d55 not found: ID does not exist" containerID="b025501f51d4056858c744e8934e7aa6e9921f9522ec6d8d81da3a79feaf0d55" Jun 08 18:18:09.759767 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.759724 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b025501f51d4056858c744e8934e7aa6e9921f9522ec6d8d81da3a79feaf0d55"} err="failed to get container status \"b025501f51d4056858c744e8934e7aa6e9921f9522ec6d8d81da3a79feaf0d55\": rpc error: code = NotFound desc = could not find container \"b025501f51d4056858c744e8934e7aa6e9921f9522ec6d8d81da3a79feaf0d55\": container with ID starting with b025501f51d4056858c744e8934e7aa6e9921f9522ec6d8d81da3a79feaf0d55 not found: ID does not exist" Jun 08 18:18:09.765009 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.764990 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 18:18:09.765229 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.765216 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79eae03b-1bf3-40dd-adfd-be1fd34224ae" containerName="registry" Jun 08 18:18:09.765273 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.765231 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="79eae03b-1bf3-40dd-adfd-be1fd34224ae" containerName="registry" Jun 08 18:18:09.765273 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.765240 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="kube-rbac-proxy-web" Jun 08 18:18:09.765273 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.765246 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="kube-rbac-proxy-web" Jun 08 18:18:09.765273 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.765259 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="prometheus" Jun 08 18:18:09.765273 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.765265 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="prometheus" Jun 08 18:18:09.765273 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.765272 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="thanos-sidecar" Jun 08 18:18:09.765439 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.765277 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="thanos-sidecar" Jun 08 18:18:09.765439 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.765290 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="kube-rbac-proxy" Jun 08 18:18:09.765439 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.765296 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="kube-rbac-proxy" Jun 08 18:18:09.765439 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.765306 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="init-config-reloader" Jun 08 18:18:09.765439 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.765313 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="init-config-reloader" Jun 08 18:18:09.765439 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.765319 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="config-reloader" Jun 08 18:18:09.765439 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.765324 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="config-reloader" Jun 08 18:18:09.765439 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.765330 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="kube-rbac-proxy-thanos" Jun 08 18:18:09.765439 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.765335 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="kube-rbac-proxy-thanos" Jun 08 18:18:09.765439 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.765377 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="config-reloader" Jun 08 18:18:09.765439 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.765385 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="kube-rbac-proxy-web" Jun 08 18:18:09.765439 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.765390 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="kube-rbac-proxy" Jun 08 18:18:09.765439 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.765396 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="thanos-sidecar" Jun 08 18:18:09.765439 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.765400 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="kube-rbac-proxy-thanos" Jun 08 18:18:09.765439 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.765407 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" containerName="prometheus" Jun 08 18:18:09.765439 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.765412 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="79eae03b-1bf3-40dd-adfd-be1fd34224ae" containerName="registry" Jun 08 18:18:09.768572 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.768555 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.770697 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.770676 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 08 18:18:09.770814 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.770801 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 08 18:18:09.771034 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.771014 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 08 18:18:09.771034 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.771024 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 08 18:18:09.771195 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.771085 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-859zp\"" Jun 08 18:18:09.771195 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.771101 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 08 18:18:09.771195 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.771085 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 08 18:18:09.771371 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.771356 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 08 18:18:09.771431 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.771389 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 08 18:18:09.771479 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.771457 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 08 18:18:09.771537 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.771481 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 08 18:18:09.771537 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.771484 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6qur38cbugthe\"" Jun 08 18:18:09.772133 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.772115 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 08 18:18:09.772224 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.772144 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 08 18:18:09.772224 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.772178 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 08 18:18:09.775930 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.775909 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 08 18:18:09.778878 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.778862 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 08 18:18:09.788709 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.788689 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 18:18:09.844513 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.844487 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dd75e56a-8538-4f11-8a9d-b011e66cc670-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.844627 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.844518 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd75e56a-8538-4f11-8a9d-b011e66cc670-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.844627 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.844539 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.844627 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.844557 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dd75e56a-8538-4f11-8a9d-b011e66cc670-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.844627 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.844611 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-web-config\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.844791 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.844640 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.844791 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.844684 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd75e56a-8538-4f11-8a9d-b011e66cc670-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.844791 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.844722 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-config\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.844791 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.844747 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dd75e56a-8538-4f11-8a9d-b011e66cc670-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.844791 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.844761 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd75e56a-8538-4f11-8a9d-b011e66cc670-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.844791 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.844788 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dd75e56a-8538-4f11-8a9d-b011e66cc670-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.845087 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.844819 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.845087 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.844836 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.845087 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.844850 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dd75e56a-8538-4f11-8a9d-b011e66cc670-config-out\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.845087 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.844870 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.845087 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.844924 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.845087 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.844981 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.845087 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.845010 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd75e56a-8538-4f11-8a9d-b011e66cc670-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.845087 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.845040 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dd75e56a-8538-4f11-8a9d-b011e66cc670-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.845087 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.845070 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwkxs\" (UniqueName: \"kubernetes.io/projected/dd75e56a-8538-4f11-8a9d-b011e66cc670-kube-api-access-xwkxs\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.945578 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.945552 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.945731 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.945587 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dd75e56a-8538-4f11-8a9d-b011e66cc670-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.945731 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.945605 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-web-config\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.945731 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.945620 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.945731 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.945636 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd75e56a-8538-4f11-8a9d-b011e66cc670-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.945731 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.945659 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-config\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.945731 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.945693 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dd75e56a-8538-4f11-8a9d-b011e66cc670-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.945731 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.945716 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd75e56a-8538-4f11-8a9d-b011e66cc670-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.946169 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.945754 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dd75e56a-8538-4f11-8a9d-b011e66cc670-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.946169 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.945786 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.946169 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.945812 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.946169 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.945833 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dd75e56a-8538-4f11-8a9d-b011e66cc670-config-out\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.946169 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.945855 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.946169 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.945878 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.946169 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.945896 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.946169 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.945910 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd75e56a-8538-4f11-8a9d-b011e66cc670-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.946169 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.945929 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dd75e56a-8538-4f11-8a9d-b011e66cc670-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.946169 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.945947 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwkxs\" (UniqueName: \"kubernetes.io/projected/dd75e56a-8538-4f11-8a9d-b011e66cc670-kube-api-access-xwkxs\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.946169 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.946002 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dd75e56a-8538-4f11-8a9d-b011e66cc670-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.946169 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.946027 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd75e56a-8538-4f11-8a9d-b011e66cc670-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.946738 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.946280 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dd75e56a-8538-4f11-8a9d-b011e66cc670-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.946738 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.946421 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd75e56a-8538-4f11-8a9d-b011e66cc670-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.946738 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.946487 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd75e56a-8538-4f11-8a9d-b011e66cc670-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.946738 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.946622 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dd75e56a-8538-4f11-8a9d-b011e66cc670-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.947567 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.947316 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd75e56a-8538-4f11-8a9d-b011e66cc670-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.949412 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.949379 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-web-config\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.949515 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.949498 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.949580 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.949512 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-config\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.949580 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.949540 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.949580 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.949544 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.949726 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.949645 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dd75e56a-8538-4f11-8a9d-b011e66cc670-config-out\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.950106 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.950054 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dd75e56a-8538-4f11-8a9d-b011e66cc670-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.950283 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.950254 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.950385 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.950327 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd75e56a-8538-4f11-8a9d-b011e66cc670-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.950446 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.950425 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dd75e56a-8538-4f11-8a9d-b011e66cc670-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.951620 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.951599 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.951738 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.951720 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dd75e56a-8538-4f11-8a9d-b011e66cc670-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.951792 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.951754 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.951792 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.951762 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dd75e56a-8538-4f11-8a9d-b011e66cc670-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:09.957874 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:09.957850 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwkxs\" (UniqueName: \"kubernetes.io/projected/dd75e56a-8538-4f11-8a9d-b011e66cc670-kube-api-access-xwkxs\") pod \"prometheus-k8s-0\" (UID: \"dd75e56a-8538-4f11-8a9d-b011e66cc670\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:10.079015 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:10.078973 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:18:10.230659 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:10.230581 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 18:18:10.234027 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:18:10.233997 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd75e56a_8538_4f11_8a9d_b011e66cc670.slice/crio-66334d37b6008d54cd29bea5461341ae7439599777c750659ecd19380bf2793d WatchSource:0}: Error finding container 66334d37b6008d54cd29bea5461341ae7439599777c750659ecd19380bf2793d: Status 404 returned error can't find the container with id 66334d37b6008d54cd29bea5461341ae7439599777c750659ecd19380bf2793d Jun 08 18:18:10.594419 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:10.594346 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d3bea65-1ded-49c4-a382-283bf76cda67" path="/var/lib/kubelet/pods/7d3bea65-1ded-49c4-a382-283bf76cda67/volumes" Jun 08 18:18:10.714921 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:10.714883 2572 generic.go:356] "Generic (PLEG): container finished" podID="dd75e56a-8538-4f11-8a9d-b011e66cc670" containerID="02b5e7dd8bdb18d18e7118bcea5e82905e88890b42cbdcc4422747ff1b65d991" exitCode=0 Jun 08 18:18:10.715110 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:10.714983 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd75e56a-8538-4f11-8a9d-b011e66cc670","Type":"ContainerDied","Data":"02b5e7dd8bdb18d18e7118bcea5e82905e88890b42cbdcc4422747ff1b65d991"} Jun 08 18:18:10.715110 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:10.715061 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd75e56a-8538-4f11-8a9d-b011e66cc670","Type":"ContainerStarted","Data":"66334d37b6008d54cd29bea5461341ae7439599777c750659ecd19380bf2793d"} Jun 08 18:18:11.722651 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:11.722615 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd75e56a-8538-4f11-8a9d-b011e66cc670","Type":"ContainerStarted","Data":"0424d925ffcab3268d58d37635848ceffa5401173e84ab4500a8b0448f45078b"} Jun 08 18:18:11.723033 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:11.722685 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd75e56a-8538-4f11-8a9d-b011e66cc670","Type":"ContainerStarted","Data":"8e024f2d2106577f29e55bc8beac87c75df9542c121f85df403145835b7ccf25"} Jun 08 18:18:11.723033 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:11.722718 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd75e56a-8538-4f11-8a9d-b011e66cc670","Type":"ContainerStarted","Data":"5da871526adae14cd1278bd2adec7af29f7eeb0d56455140be961195a0da7092"} Jun 08 18:18:11.723033 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:11.722744 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd75e56a-8538-4f11-8a9d-b011e66cc670","Type":"ContainerStarted","Data":"dd74924fce2104d07ab3c5c3eeaafcb4b89dc9cc7f07d2ae225e2cf34366f83f"} Jun 08 18:18:11.723033 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:11.722773 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd75e56a-8538-4f11-8a9d-b011e66cc670","Type":"ContainerStarted","Data":"d504bcaba38003b4def6b11fdbc52e60d823ea33ffaf102ace6c14e640821c4a"} Jun 08 18:18:11.723033 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:11.722796 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd75e56a-8538-4f11-8a9d-b011e66cc670","Type":"ContainerStarted","Data":"0b1f2ffb38b6dcf9742b7474b422ec7b2de6ec996fd099c12e474db3141e13d4"} Jun 08 18:18:11.751109 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:11.751067 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.730854876 podStartE2EDuration="2.751053609s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.198733ms" isStatefulPod=false podCreationTimestamp="2026-06-08 18:18:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:18:11.749837515 +0000 UTC m=+325.604344334" watchObservedRunningTime="2026-06-08 18:18:11.751053609 +0000 UTC m=+325.605560406" Jun 08 18:18:15.079304 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:18:15.079245 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:19:10.079570 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:19:10.079529 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:19:10.095151 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:19:10.095124 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:19:10.945436 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:19:10.945408 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 18:22:46.564090 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:22:46.564057 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 18:22:46.565251 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:22:46.565230 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 18:27:46.584598 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:27:46.584521 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 18:27:46.587087 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:27:46.587064 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 18:32:46.607716 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:32:46.607689 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 18:32:46.609847 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:32:46.609827 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 18:34:09.600819 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.600787 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh"] Jun 08 18:34:09.604084 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.604066 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" Jun 08 18:34:09.607376 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.607355 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 08 18:34:09.607376 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.607364 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 08 18:34:09.607547 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.607427 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hms6n\"" Jun 08 18:34:09.607547 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.607361 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 08 18:34:09.607712 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.607699 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 08 18:34:09.625128 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.625103 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh"] Jun 08 18:34:09.758172 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.758132 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg6gj\" (UniqueName: \"kubernetes.io/projected/32ac2387-9104-4c1a-a52f-74e10558d6a8-kube-api-access-zg6gj\") pod \"opendatahub-operator-controller-manager-9564c99bf-xf4nh\" (UID: \"32ac2387-9104-4c1a-a52f-74e10558d6a8\") " pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" Jun 08 18:34:09.758172 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.758178 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/32ac2387-9104-4c1a-a52f-74e10558d6a8-manifests\") pod \"opendatahub-operator-controller-manager-9564c99bf-xf4nh\" (UID: \"32ac2387-9104-4c1a-a52f-74e10558d6a8\") " pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" Jun 08 18:34:09.758440 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.758280 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/32ac2387-9104-4c1a-a52f-74e10558d6a8-webhook-cert\") pod \"opendatahub-operator-controller-manager-9564c99bf-xf4nh\" (UID: \"32ac2387-9104-4c1a-a52f-74e10558d6a8\") " pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" Jun 08 18:34:09.758440 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.758340 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32ac2387-9104-4c1a-a52f-74e10558d6a8-tmp\") pod \"opendatahub-operator-controller-manager-9564c99bf-xf4nh\" (UID: \"32ac2387-9104-4c1a-a52f-74e10558d6a8\") " pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" Jun 08 18:34:09.758440 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.758368 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/32ac2387-9104-4c1a-a52f-74e10558d6a8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9564c99bf-xf4nh\" (UID: \"32ac2387-9104-4c1a-a52f-74e10558d6a8\") " pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" Jun 08 18:34:09.859265 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.859184 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg6gj\" (UniqueName: \"kubernetes.io/projected/32ac2387-9104-4c1a-a52f-74e10558d6a8-kube-api-access-zg6gj\") pod \"opendatahub-operator-controller-manager-9564c99bf-xf4nh\" (UID: \"32ac2387-9104-4c1a-a52f-74e10558d6a8\") " pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" Jun 08 18:34:09.859265 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.859226 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/32ac2387-9104-4c1a-a52f-74e10558d6a8-manifests\") pod \"opendatahub-operator-controller-manager-9564c99bf-xf4nh\" (UID: \"32ac2387-9104-4c1a-a52f-74e10558d6a8\") " pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" Jun 08 18:34:09.859265 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.859263 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/32ac2387-9104-4c1a-a52f-74e10558d6a8-webhook-cert\") pod \"opendatahub-operator-controller-manager-9564c99bf-xf4nh\" (UID: \"32ac2387-9104-4c1a-a52f-74e10558d6a8\") " pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" Jun 08 18:34:09.859507 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.859296 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32ac2387-9104-4c1a-a52f-74e10558d6a8-tmp\") pod \"opendatahub-operator-controller-manager-9564c99bf-xf4nh\" (UID: \"32ac2387-9104-4c1a-a52f-74e10558d6a8\") " pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" Jun 08 18:34:09.859507 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.859311 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/32ac2387-9104-4c1a-a52f-74e10558d6a8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9564c99bf-xf4nh\" (UID: \"32ac2387-9104-4c1a-a52f-74e10558d6a8\") " pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" Jun 08 18:34:09.859603 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.859572 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/32ac2387-9104-4c1a-a52f-74e10558d6a8-manifests\") pod \"opendatahub-operator-controller-manager-9564c99bf-xf4nh\" (UID: \"32ac2387-9104-4c1a-a52f-74e10558d6a8\") " pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" Jun 08 18:34:09.861484 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.861451 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32ac2387-9104-4c1a-a52f-74e10558d6a8-tmp\") pod \"opendatahub-operator-controller-manager-9564c99bf-xf4nh\" (UID: \"32ac2387-9104-4c1a-a52f-74e10558d6a8\") " pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" Jun 08 18:34:09.861797 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.861775 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/32ac2387-9104-4c1a-a52f-74e10558d6a8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9564c99bf-xf4nh\" (UID: \"32ac2387-9104-4c1a-a52f-74e10558d6a8\") " pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" Jun 08 18:34:09.861837 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.861775 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/32ac2387-9104-4c1a-a52f-74e10558d6a8-webhook-cert\") pod \"opendatahub-operator-controller-manager-9564c99bf-xf4nh\" (UID: \"32ac2387-9104-4c1a-a52f-74e10558d6a8\") " pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" Jun 08 18:34:09.871537 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.871510 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg6gj\" (UniqueName: \"kubernetes.io/projected/32ac2387-9104-4c1a-a52f-74e10558d6a8-kube-api-access-zg6gj\") pod \"opendatahub-operator-controller-manager-9564c99bf-xf4nh\" (UID: \"32ac2387-9104-4c1a-a52f-74e10558d6a8\") " pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" Jun 08 18:34:09.914226 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:09.914186 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" Jun 08 18:34:10.063238 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:10.063202 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh"] Jun 08 18:34:10.066484 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:34:10.066458 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32ac2387_9104_4c1a_a52f_74e10558d6a8.slice/crio-bd61ac429c8ed3fe3bbba051c5230e2b199f923a8fff6d2ac17f350a845dc3c2 WatchSource:0}: Error finding container bd61ac429c8ed3fe3bbba051c5230e2b199f923a8fff6d2ac17f350a845dc3c2: Status 404 returned error can't find the container with id bd61ac429c8ed3fe3bbba051c5230e2b199f923a8fff6d2ac17f350a845dc3c2 Jun 08 18:34:10.068184 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:10.068168 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 18:34:11.075819 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:11.075726 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" event={"ID":"32ac2387-9104-4c1a-a52f-74e10558d6a8","Type":"ContainerStarted","Data":"bd61ac429c8ed3fe3bbba051c5230e2b199f923a8fff6d2ac17f350a845dc3c2"} Jun 08 18:34:14.090867 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:14.090829 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" event={"ID":"32ac2387-9104-4c1a-a52f-74e10558d6a8","Type":"ContainerStarted","Data":"f49951ef5c1552e762adb9ab2305d5139db4557cd5ceef448ee9c6e956af0a1a"} Jun 08 18:34:15.096362 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:15.096327 2572 generic.go:356] "Generic (PLEG): container finished" podID="32ac2387-9104-4c1a-a52f-74e10558d6a8" containerID="f49951ef5c1552e762adb9ab2305d5139db4557cd5ceef448ee9c6e956af0a1a" exitCode=0 Jun 08 18:34:15.096713 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:15.096371 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" event={"ID":"32ac2387-9104-4c1a-a52f-74e10558d6a8","Type":"ContainerDied","Data":"f49951ef5c1552e762adb9ab2305d5139db4557cd5ceef448ee9c6e956af0a1a"} Jun 08 18:34:16.102152 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:16.102118 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" event={"ID":"32ac2387-9104-4c1a-a52f-74e10558d6a8","Type":"ContainerStarted","Data":"b71d9789bed5d8092d5517706f72b423124a1f765ee52d61b51e58598b24e49e"} Jun 08 18:34:16.102547 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:16.102211 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" Jun 08 18:34:16.125426 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:16.125366 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" podStartSLOduration=2.705057135 podStartE2EDuration="7.125345184s" totalImagesPullingTime="3.455153008s" totalInitContainerRuntime="965.135041ms" isStatefulPod=false podCreationTimestamp="2026-06-08 18:34:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:34:16.123624226 +0000 UTC m=+1289.978131023" watchObservedRunningTime="2026-06-08 18:34:16.125345184 +0000 UTC m=+1289.979851984" Jun 08 18:34:27.107864 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:27.107833 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9564c99bf-xf4nh" Jun 08 18:34:29.503683 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:29.503645 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-ccd6bbfcc-5qb5d"] Jun 08 18:34:29.512032 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:29.512009 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-ccd6bbfcc-5qb5d" Jun 08 18:34:29.514765 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:29.514744 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 08 18:34:29.514993 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:29.514979 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 08 18:34:29.543531 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:29.543505 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-ccd6bbfcc-5qb5d"] Jun 08 18:34:29.613469 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:29.613439 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36615221-c076-49fc-8c35-77476b334f68-tmp\") pod \"kube-auth-proxy-ccd6bbfcc-5qb5d\" (UID: \"36615221-c076-49fc-8c35-77476b334f68\") " pod="openshift-ingress/kube-auth-proxy-ccd6bbfcc-5qb5d" Jun 08 18:34:29.613646 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:29.613477 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36615221-c076-49fc-8c35-77476b334f68-tls-certs\") pod \"kube-auth-proxy-ccd6bbfcc-5qb5d\" (UID: \"36615221-c076-49fc-8c35-77476b334f68\") " pod="openshift-ingress/kube-auth-proxy-ccd6bbfcc-5qb5d" Jun 08 18:34:29.613646 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:29.613591 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d2w9\" (UniqueName: \"kubernetes.io/projected/36615221-c076-49fc-8c35-77476b334f68-kube-api-access-9d2w9\") pod \"kube-auth-proxy-ccd6bbfcc-5qb5d\" (UID: \"36615221-c076-49fc-8c35-77476b334f68\") " pod="openshift-ingress/kube-auth-proxy-ccd6bbfcc-5qb5d" Jun 08 18:34:29.714266 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:29.714234 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36615221-c076-49fc-8c35-77476b334f68-tmp\") pod \"kube-auth-proxy-ccd6bbfcc-5qb5d\" (UID: \"36615221-c076-49fc-8c35-77476b334f68\") " pod="openshift-ingress/kube-auth-proxy-ccd6bbfcc-5qb5d" Jun 08 18:34:29.714266 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:29.714270 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36615221-c076-49fc-8c35-77476b334f68-tls-certs\") pod \"kube-auth-proxy-ccd6bbfcc-5qb5d\" (UID: \"36615221-c076-49fc-8c35-77476b334f68\") " pod="openshift-ingress/kube-auth-proxy-ccd6bbfcc-5qb5d" Jun 08 18:34:29.714524 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:29.714312 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d2w9\" (UniqueName: \"kubernetes.io/projected/36615221-c076-49fc-8c35-77476b334f68-kube-api-access-9d2w9\") pod \"kube-auth-proxy-ccd6bbfcc-5qb5d\" (UID: \"36615221-c076-49fc-8c35-77476b334f68\") " pod="openshift-ingress/kube-auth-proxy-ccd6bbfcc-5qb5d" Jun 08 18:34:29.714524 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:34:29.714409 2572 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jun 08 18:34:29.714524 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:34:29.714498 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36615221-c076-49fc-8c35-77476b334f68-tls-certs podName:36615221-c076-49fc-8c35-77476b334f68 nodeName:}" failed. No retries permitted until 2026-06-08 18:34:30.214473631 +0000 UTC m=+1304.068980423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/36615221-c076-49fc-8c35-77476b334f68-tls-certs") pod "kube-auth-proxy-ccd6bbfcc-5qb5d" (UID: "36615221-c076-49fc-8c35-77476b334f68") : secret "kube-auth-proxy-tls" not found Jun 08 18:34:29.716412 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:29.716393 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36615221-c076-49fc-8c35-77476b334f68-tmp\") pod \"kube-auth-proxy-ccd6bbfcc-5qb5d\" (UID: \"36615221-c076-49fc-8c35-77476b334f68\") " pod="openshift-ingress/kube-auth-proxy-ccd6bbfcc-5qb5d" Jun 08 18:34:29.741430 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:29.741407 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d2w9\" (UniqueName: \"kubernetes.io/projected/36615221-c076-49fc-8c35-77476b334f68-kube-api-access-9d2w9\") pod \"kube-auth-proxy-ccd6bbfcc-5qb5d\" (UID: \"36615221-c076-49fc-8c35-77476b334f68\") " pod="openshift-ingress/kube-auth-proxy-ccd6bbfcc-5qb5d" Jun 08 18:34:29.806134 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:29.806050 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-mqfxx"] Jun 08 18:34:29.808289 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:29.808263 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-mqfxx" Jun 08 18:34:29.810726 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:29.810705 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jun 08 18:34:29.810849 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:29.810708 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-lspq8\"" Jun 08 18:34:29.821457 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:29.821435 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-mqfxx"] Jun 08 18:34:29.915930 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:29.915893 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5976\" (UniqueName: \"kubernetes.io/projected/7b12b047-9e68-4273-a445-bfeeb6e06cbd-kube-api-access-d5976\") pod \"odh-model-controller-6dc94f689f-mqfxx\" (UID: \"7b12b047-9e68-4273-a445-bfeeb6e06cbd\") " pod="opendatahub/odh-model-controller-6dc94f689f-mqfxx" Jun 08 18:34:29.916127 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:29.915951 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b12b047-9e68-4273-a445-bfeeb6e06cbd-cert\") pod \"odh-model-controller-6dc94f689f-mqfxx\" (UID: \"7b12b047-9e68-4273-a445-bfeeb6e06cbd\") " pod="opendatahub/odh-model-controller-6dc94f689f-mqfxx" Jun 08 18:34:30.016621 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:30.016582 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b12b047-9e68-4273-a445-bfeeb6e06cbd-cert\") pod \"odh-model-controller-6dc94f689f-mqfxx\" (UID: \"7b12b047-9e68-4273-a445-bfeeb6e06cbd\") " pod="opendatahub/odh-model-controller-6dc94f689f-mqfxx" Jun 08 18:34:30.016804 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:30.016691 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5976\" (UniqueName: \"kubernetes.io/projected/7b12b047-9e68-4273-a445-bfeeb6e06cbd-kube-api-access-d5976\") pod \"odh-model-controller-6dc94f689f-mqfxx\" (UID: \"7b12b047-9e68-4273-a445-bfeeb6e06cbd\") " pod="opendatahub/odh-model-controller-6dc94f689f-mqfxx" Jun 08 18:34:30.016804 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:34:30.016735 2572 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jun 08 18:34:30.016804 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:34:30.016801 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b12b047-9e68-4273-a445-bfeeb6e06cbd-cert podName:7b12b047-9e68-4273-a445-bfeeb6e06cbd nodeName:}" failed. No retries permitted until 2026-06-08 18:34:30.516784804 +0000 UTC m=+1304.371291585 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7b12b047-9e68-4273-a445-bfeeb6e06cbd-cert") pod "odh-model-controller-6dc94f689f-mqfxx" (UID: "7b12b047-9e68-4273-a445-bfeeb6e06cbd") : secret "odh-model-controller-webhook-cert" not found Jun 08 18:34:30.029410 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:30.029384 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5976\" (UniqueName: \"kubernetes.io/projected/7b12b047-9e68-4273-a445-bfeeb6e06cbd-kube-api-access-d5976\") pod \"odh-model-controller-6dc94f689f-mqfxx\" (UID: \"7b12b047-9e68-4273-a445-bfeeb6e06cbd\") " pod="opendatahub/odh-model-controller-6dc94f689f-mqfxx" Jun 08 18:34:30.219136 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:30.219102 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36615221-c076-49fc-8c35-77476b334f68-tls-certs\") pod \"kube-auth-proxy-ccd6bbfcc-5qb5d\" (UID: \"36615221-c076-49fc-8c35-77476b334f68\") " pod="openshift-ingress/kube-auth-proxy-ccd6bbfcc-5qb5d" Jun 08 18:34:30.221529 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:30.221497 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36615221-c076-49fc-8c35-77476b334f68-tls-certs\") pod \"kube-auth-proxy-ccd6bbfcc-5qb5d\" (UID: \"36615221-c076-49fc-8c35-77476b334f68\") " pod="openshift-ingress/kube-auth-proxy-ccd6bbfcc-5qb5d" Jun 08 18:34:30.420936 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:30.420890 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-ccd6bbfcc-5qb5d" Jun 08 18:34:30.521566 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:30.521527 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b12b047-9e68-4273-a445-bfeeb6e06cbd-cert\") pod \"odh-model-controller-6dc94f689f-mqfxx\" (UID: \"7b12b047-9e68-4273-a445-bfeeb6e06cbd\") " pod="opendatahub/odh-model-controller-6dc94f689f-mqfxx" Jun 08 18:34:30.524325 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:30.524296 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b12b047-9e68-4273-a445-bfeeb6e06cbd-cert\") pod \"odh-model-controller-6dc94f689f-mqfxx\" (UID: \"7b12b047-9e68-4273-a445-bfeeb6e06cbd\") " pod="opendatahub/odh-model-controller-6dc94f689f-mqfxx" Jun 08 18:34:30.568108 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:30.568083 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-ccd6bbfcc-5qb5d"] Jun 08 18:34:30.570028 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:34:30.569996 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36615221_c076_49fc_8c35_77476b334f68.slice/crio-096652346bdc70340cd4948469f29355e15a32d37f01a692a470a41c496c9324 WatchSource:0}: Error finding container 096652346bdc70340cd4948469f29355e15a32d37f01a692a470a41c496c9324: Status 404 returned error can't find the container with id 096652346bdc70340cd4948469f29355e15a32d37f01a692a470a41c496c9324 Jun 08 18:34:30.660737 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:30.660699 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-nplwv"] Jun 08 18:34:30.663780 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:30.663756 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-nplwv" Jun 08 18:34:30.666906 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:30.666889 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jun 08 18:34:30.667023 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:30.666888 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-rqc5s\"" Jun 08 18:34:30.680990 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:30.680948 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-nplwv"] Jun 08 18:34:30.719082 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:30.719050 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-mqfxx" Jun 08 18:34:30.723390 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:30.723364 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxvqf\" (UniqueName: \"kubernetes.io/projected/52466ef3-77e8-45f8-ad25-f1f1b5612059-kube-api-access-kxvqf\") pod \"model-serving-api-5dd488fc48-nplwv\" (UID: \"52466ef3-77e8-45f8-ad25-f1f1b5612059\") " pod="opendatahub/model-serving-api-5dd488fc48-nplwv" Jun 08 18:34:30.723531 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:30.723432 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52466ef3-77e8-45f8-ad25-f1f1b5612059-tls-certs\") pod \"model-serving-api-5dd488fc48-nplwv\" (UID: \"52466ef3-77e8-45f8-ad25-f1f1b5612059\") " pod="opendatahub/model-serving-api-5dd488fc48-nplwv" Jun 08 18:34:30.824665 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:30.824631 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52466ef3-77e8-45f8-ad25-f1f1b5612059-tls-certs\") pod \"model-serving-api-5dd488fc48-nplwv\" (UID: \"52466ef3-77e8-45f8-ad25-f1f1b5612059\") " pod="opendatahub/model-serving-api-5dd488fc48-nplwv" Jun 08 18:34:30.824827 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:30.824739 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxvqf\" (UniqueName: \"kubernetes.io/projected/52466ef3-77e8-45f8-ad25-f1f1b5612059-kube-api-access-kxvqf\") pod \"model-serving-api-5dd488fc48-nplwv\" (UID: \"52466ef3-77e8-45f8-ad25-f1f1b5612059\") " pod="opendatahub/model-serving-api-5dd488fc48-nplwv" Jun 08 18:34:30.827113 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:30.827089 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52466ef3-77e8-45f8-ad25-f1f1b5612059-tls-certs\") pod \"model-serving-api-5dd488fc48-nplwv\" (UID: \"52466ef3-77e8-45f8-ad25-f1f1b5612059\") " pod="opendatahub/model-serving-api-5dd488fc48-nplwv" Jun 08 18:34:30.834810 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:30.834789 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxvqf\" (UniqueName: \"kubernetes.io/projected/52466ef3-77e8-45f8-ad25-f1f1b5612059-kube-api-access-kxvqf\") pod \"model-serving-api-5dd488fc48-nplwv\" (UID: \"52466ef3-77e8-45f8-ad25-f1f1b5612059\") " pod="opendatahub/model-serving-api-5dd488fc48-nplwv" Jun 08 18:34:30.841051 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:30.841027 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-mqfxx"] Jun 08 18:34:30.843844 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:34:30.843820 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b12b047_9e68_4273_a445_bfeeb6e06cbd.slice/crio-aaf4968bb12e9ae776a8533add6550bdea4095bd31ceced889ef78bf582f0198 WatchSource:0}: Error finding container aaf4968bb12e9ae776a8533add6550bdea4095bd31ceced889ef78bf582f0198: Status 404 returned error can't find the container with id aaf4968bb12e9ae776a8533add6550bdea4095bd31ceced889ef78bf582f0198 Jun 08 18:34:30.974205 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:30.974125 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-nplwv" Jun 08 18:34:31.102143 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:31.101985 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-nplwv"] Jun 08 18:34:31.104809 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:34:31.104770 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52466ef3_77e8_45f8_ad25_f1f1b5612059.slice/crio-95e4919b744ed2ad68f72a3d8da834debd84ecc2c2211fe3baa365fdbc6f9232 WatchSource:0}: Error finding container 95e4919b744ed2ad68f72a3d8da834debd84ecc2c2211fe3baa365fdbc6f9232: Status 404 returned error can't find the container with id 95e4919b744ed2ad68f72a3d8da834debd84ecc2c2211fe3baa365fdbc6f9232 Jun 08 18:34:31.157538 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:31.157498 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-nplwv" event={"ID":"52466ef3-77e8-45f8-ad25-f1f1b5612059","Type":"ContainerStarted","Data":"95e4919b744ed2ad68f72a3d8da834debd84ecc2c2211fe3baa365fdbc6f9232"} Jun 08 18:34:31.159028 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:31.158990 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-mqfxx" event={"ID":"7b12b047-9e68-4273-a445-bfeeb6e06cbd","Type":"ContainerStarted","Data":"aaf4968bb12e9ae776a8533add6550bdea4095bd31ceced889ef78bf582f0198"} Jun 08 18:34:31.160438 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:31.160410 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-ccd6bbfcc-5qb5d" event={"ID":"36615221-c076-49fc-8c35-77476b334f68","Type":"ContainerStarted","Data":"096652346bdc70340cd4948469f29355e15a32d37f01a692a470a41c496c9324"} Jun 08 18:34:36.183453 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:36.183419 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-nplwv" event={"ID":"52466ef3-77e8-45f8-ad25-f1f1b5612059","Type":"ContainerStarted","Data":"cf4fe12b8493897158c1c533676bf6f6f2f70fbf3a7200e26609e00129061fa1"} Jun 08 18:34:36.183906 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:36.183573 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-5dd488fc48-nplwv" Jun 08 18:34:36.184913 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:36.184895 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-mqfxx" event={"ID":"7b12b047-9e68-4273-a445-bfeeb6e06cbd","Type":"ContainerStarted","Data":"9d3b678704192f746738913491b7355864414291f797d597d7fbc004e31836af"} Jun 08 18:34:36.184985 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:36.184933 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-6dc94f689f-mqfxx" Jun 08 18:34:36.186499 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:36.186475 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-ccd6bbfcc-5qb5d" event={"ID":"36615221-c076-49fc-8c35-77476b334f68","Type":"ContainerStarted","Data":"6a81dff3adf28cc216836c7f04593ada762766c5e8956a2f7fa1a699bc9552bb"} Jun 08 18:34:36.223403 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:36.223353 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-5dd488fc48-nplwv" podStartSLOduration=2.143272401 podStartE2EDuration="6.223337379s" totalImagesPullingTime="4.080064978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:34:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:34:36.221628425 +0000 UTC m=+1310.076135222" watchObservedRunningTime="2026-06-08 18:34:36.223337379 +0000 UTC m=+1310.077844177" Jun 08 18:34:36.252523 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:36.252470 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-6dc94f689f-mqfxx" podStartSLOduration=2.860509431 podStartE2EDuration="7.252456722s" totalImagesPullingTime="4.391947291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:34:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:34:36.247156279 +0000 UTC m=+1310.101663076" watchObservedRunningTime="2026-06-08 18:34:36.252456722 +0000 UTC m=+1310.106963520" Jun 08 18:34:36.280014 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:36.279950 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-ccd6bbfcc-5qb5d" podStartSLOduration=2.664533335 podStartE2EDuration="7.279933667s" totalImagesPullingTime="4.615400332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:34:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:34:36.278906958 +0000 UTC m=+1310.133413755" watchObservedRunningTime="2026-06-08 18:34:36.279933667 +0000 UTC m=+1310.134440464" Jun 08 18:34:41.505445 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:41.505398 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-pvcqw"] Jun 08 18:34:41.508652 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:41.508633 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-pvcqw" Jun 08 18:34:41.511970 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:41.511940 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jun 08 18:34:41.512090 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:41.511984 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-rx54x\"" Jun 08 18:34:41.542624 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:41.542591 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-pvcqw"] Jun 08 18:34:41.554045 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:41.554015 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-z2vmb"] Jun 08 18:34:41.557254 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:41.557232 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2vmb" Jun 08 18:34:41.561315 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:41.561293 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-9ds56\"" Jun 08 18:34:41.565084 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:41.565066 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jun 08 18:34:41.596860 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:41.596831 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-z2vmb"] Jun 08 18:34:41.622588 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:41.622557 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqf92\" (UniqueName: \"kubernetes.io/projected/53b4292e-3e7b-4c44-a681-ea0baac43d99-kube-api-access-gqf92\") pod \"kserve-controller-manager-69fc866d76-pvcqw\" (UID: \"53b4292e-3e7b-4c44-a681-ea0baac43d99\") " pod="opendatahub/kserve-controller-manager-69fc866d76-pvcqw" Jun 08 18:34:41.622588 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:41.622589 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwj6r\" (UniqueName: \"kubernetes.io/projected/c8da8b02-c7b3-4a1a-aa7c-7e35ca7fc0ce-kube-api-access-xwj6r\") pod \"llmisvc-controller-manager-864bfd7bc9-z2vmb\" (UID: \"c8da8b02-c7b3-4a1a-aa7c-7e35ca7fc0ce\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2vmb" Jun 08 18:34:41.622780 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:41.622617 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53b4292e-3e7b-4c44-a681-ea0baac43d99-cert\") pod \"kserve-controller-manager-69fc866d76-pvcqw\" (UID: \"53b4292e-3e7b-4c44-a681-ea0baac43d99\") " pod="opendatahub/kserve-controller-manager-69fc866d76-pvcqw" Jun 08 18:34:41.622780 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:41.622636 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8da8b02-c7b3-4a1a-aa7c-7e35ca7fc0ce-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-z2vmb\" (UID: \"c8da8b02-c7b3-4a1a-aa7c-7e35ca7fc0ce\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2vmb" Jun 08 18:34:41.724032 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:41.723985 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqf92\" (UniqueName: \"kubernetes.io/projected/53b4292e-3e7b-4c44-a681-ea0baac43d99-kube-api-access-gqf92\") pod \"kserve-controller-manager-69fc866d76-pvcqw\" (UID: \"53b4292e-3e7b-4c44-a681-ea0baac43d99\") " pod="opendatahub/kserve-controller-manager-69fc866d76-pvcqw" Jun 08 18:34:41.724032 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:41.724034 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwj6r\" (UniqueName: \"kubernetes.io/projected/c8da8b02-c7b3-4a1a-aa7c-7e35ca7fc0ce-kube-api-access-xwj6r\") pod \"llmisvc-controller-manager-864bfd7bc9-z2vmb\" (UID: \"c8da8b02-c7b3-4a1a-aa7c-7e35ca7fc0ce\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2vmb" Jun 08 18:34:41.724341 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:41.724065 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53b4292e-3e7b-4c44-a681-ea0baac43d99-cert\") pod \"kserve-controller-manager-69fc866d76-pvcqw\" (UID: \"53b4292e-3e7b-4c44-a681-ea0baac43d99\") " pod="opendatahub/kserve-controller-manager-69fc866d76-pvcqw" Jun 08 18:34:41.724341 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:41.724090 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8da8b02-c7b3-4a1a-aa7c-7e35ca7fc0ce-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-z2vmb\" (UID: \"c8da8b02-c7b3-4a1a-aa7c-7e35ca7fc0ce\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2vmb" Jun 08 18:34:41.724341 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:34:41.724263 2572 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jun 08 18:34:41.724341 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:34:41.724332 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8da8b02-c7b3-4a1a-aa7c-7e35ca7fc0ce-cert podName:c8da8b02-c7b3-4a1a-aa7c-7e35ca7fc0ce nodeName:}" failed. No retries permitted until 2026-06-08 18:34:42.224311734 +0000 UTC m=+1316.078818513 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c8da8b02-c7b3-4a1a-aa7c-7e35ca7fc0ce-cert") pod "llmisvc-controller-manager-864bfd7bc9-z2vmb" (UID: "c8da8b02-c7b3-4a1a-aa7c-7e35ca7fc0ce") : secret "llmisvc-webhook-server-cert" not found Jun 08 18:34:41.726602 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:41.726581 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53b4292e-3e7b-4c44-a681-ea0baac43d99-cert\") pod \"kserve-controller-manager-69fc866d76-pvcqw\" (UID: \"53b4292e-3e7b-4c44-a681-ea0baac43d99\") " pod="opendatahub/kserve-controller-manager-69fc866d76-pvcqw" Jun 08 18:34:41.746457 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:41.746430 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqf92\" (UniqueName: \"kubernetes.io/projected/53b4292e-3e7b-4c44-a681-ea0baac43d99-kube-api-access-gqf92\") pod \"kserve-controller-manager-69fc866d76-pvcqw\" (UID: \"53b4292e-3e7b-4c44-a681-ea0baac43d99\") " pod="opendatahub/kserve-controller-manager-69fc866d76-pvcqw" Jun 08 18:34:41.746581 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:41.746555 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwj6r\" (UniqueName: \"kubernetes.io/projected/c8da8b02-c7b3-4a1a-aa7c-7e35ca7fc0ce-kube-api-access-xwj6r\") pod \"llmisvc-controller-manager-864bfd7bc9-z2vmb\" (UID: \"c8da8b02-c7b3-4a1a-aa7c-7e35ca7fc0ce\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2vmb" Jun 08 18:34:41.818618 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:41.818526 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-pvcqw" Jun 08 18:34:41.960654 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:41.960628 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-pvcqw"] Jun 08 18:34:41.962835 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:34:41.962806 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53b4292e_3e7b_4c44_a681_ea0baac43d99.slice/crio-8210af9e27fa8f198e1a3c3f861f6db1cf725e4fd978f4d947c4e1204e2a9050 WatchSource:0}: Error finding container 8210af9e27fa8f198e1a3c3f861f6db1cf725e4fd978f4d947c4e1204e2a9050: Status 404 returned error can't find the container with id 8210af9e27fa8f198e1a3c3f861f6db1cf725e4fd978f4d947c4e1204e2a9050 Jun 08 18:34:42.211144 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:42.211109 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-pvcqw" event={"ID":"53b4292e-3e7b-4c44-a681-ea0baac43d99","Type":"ContainerStarted","Data":"8210af9e27fa8f198e1a3c3f861f6db1cf725e4fd978f4d947c4e1204e2a9050"} Jun 08 18:34:42.230761 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:42.230737 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8da8b02-c7b3-4a1a-aa7c-7e35ca7fc0ce-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-z2vmb\" (UID: \"c8da8b02-c7b3-4a1a-aa7c-7e35ca7fc0ce\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2vmb" Jun 08 18:34:42.233090 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:42.233073 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8da8b02-c7b3-4a1a-aa7c-7e35ca7fc0ce-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-z2vmb\" (UID: \"c8da8b02-c7b3-4a1a-aa7c-7e35ca7fc0ce\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2vmb" Jun 08 18:34:42.467106 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:42.467032 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2vmb" Jun 08 18:34:42.604457 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:42.604429 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-z2vmb"] Jun 08 18:34:42.606466 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:34:42.606423 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc8da8b02_c7b3_4a1a_aa7c_7e35ca7fc0ce.slice/crio-32a2429abee3d34cb1ef634b58c23ef45edf8d639da2cdfdcbae9bdaca3d70b8 WatchSource:0}: Error finding container 32a2429abee3d34cb1ef634b58c23ef45edf8d639da2cdfdcbae9bdaca3d70b8: Status 404 returned error can't find the container with id 32a2429abee3d34cb1ef634b58c23ef45edf8d639da2cdfdcbae9bdaca3d70b8 Jun 08 18:34:43.217211 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:43.217173 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2vmb" event={"ID":"c8da8b02-c7b3-4a1a-aa7c-7e35ca7fc0ce","Type":"ContainerStarted","Data":"32a2429abee3d34cb1ef634b58c23ef45edf8d639da2cdfdcbae9bdaca3d70b8"} Jun 08 18:34:45.228122 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:45.228084 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-pvcqw" event={"ID":"53b4292e-3e7b-4c44-a681-ea0baac43d99","Type":"ContainerStarted","Data":"56daaf8463aed1a6489f49ab87cdfc87270897a8d5765425b8eb7de93184bd81"} Jun 08 18:34:45.228497 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:45.228219 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-69fc866d76-pvcqw" Jun 08 18:34:45.278474 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:45.278423 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-69fc866d76-pvcqw" podStartSLOduration=1.6132545299999999 podStartE2EDuration="4.278406812s" totalImagesPullingTime="2.665152282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:34:45.278184808 +0000 UTC m=+1319.132691607" watchObservedRunningTime="2026-06-08 18:34:45.278406812 +0000 UTC m=+1319.132913610" Jun 08 18:34:47.192788 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:47.192759 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-6dc94f689f-mqfxx" Jun 08 18:34:47.194910 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:47.194890 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-5dd488fc48-nplwv" Jun 08 18:34:47.237343 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:47.237311 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2vmb" event={"ID":"c8da8b02-c7b3-4a1a-aa7c-7e35ca7fc0ce","Type":"ContainerStarted","Data":"bf9803ce80c1ea0c36f5eae66c37d8f4664a97506a70896bc39f26d798f6ea6a"} Jun 08 18:34:47.237513 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:47.237464 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2vmb" Jun 08 18:34:47.436947 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:47.436890 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2vmb" podStartSLOduration=2.84635166 podStartE2EDuration="6.436874193s" totalImagesPullingTime="3.590522533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:34:47.383021067 +0000 UTC m=+1321.237527864" watchObservedRunningTime="2026-06-08 18:34:47.436874193 +0000 UTC m=+1321.291381036" Jun 08 18:34:48.662163 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:48.662121 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-bjm6s"] Jun 08 18:34:48.667046 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:48.667022 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-bjm6s" Jun 08 18:34:48.669571 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:48.669550 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" Jun 08 18:34:48.670638 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:48.670619 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" Jun 08 18:34:48.670731 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:48.670623 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-5vsc6\"" Jun 08 18:34:48.686088 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:48.686065 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-bjm6s"] Jun 08 18:34:48.789289 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:48.789259 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/af5833ca-6986-401a-a984-d253b6dbc2b2-operator-config\") pod \"servicemesh-operator3-57f65f65fb-bjm6s\" (UID: \"af5833ca-6986-401a-a984-d253b6dbc2b2\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-bjm6s" Jun 08 18:34:48.789457 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:48.789309 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9pj4\" (UniqueName: \"kubernetes.io/projected/af5833ca-6986-401a-a984-d253b6dbc2b2-kube-api-access-d9pj4\") pod \"servicemesh-operator3-57f65f65fb-bjm6s\" (UID: \"af5833ca-6986-401a-a984-d253b6dbc2b2\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-bjm6s" Jun 08 18:34:48.890744 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:48.890712 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/af5833ca-6986-401a-a984-d253b6dbc2b2-operator-config\") pod \"servicemesh-operator3-57f65f65fb-bjm6s\" (UID: \"af5833ca-6986-401a-a984-d253b6dbc2b2\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-bjm6s" Jun 08 18:34:48.890890 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:48.890756 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9pj4\" (UniqueName: \"kubernetes.io/projected/af5833ca-6986-401a-a984-d253b6dbc2b2-kube-api-access-d9pj4\") pod \"servicemesh-operator3-57f65f65fb-bjm6s\" (UID: \"af5833ca-6986-401a-a984-d253b6dbc2b2\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-bjm6s" Jun 08 18:34:48.893236 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:48.893209 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/af5833ca-6986-401a-a984-d253b6dbc2b2-operator-config\") pod \"servicemesh-operator3-57f65f65fb-bjm6s\" (UID: \"af5833ca-6986-401a-a984-d253b6dbc2b2\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-bjm6s" Jun 08 18:34:48.902640 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:48.902615 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9pj4\" (UniqueName: \"kubernetes.io/projected/af5833ca-6986-401a-a984-d253b6dbc2b2-kube-api-access-d9pj4\") pod \"servicemesh-operator3-57f65f65fb-bjm6s\" (UID: \"af5833ca-6986-401a-a984-d253b6dbc2b2\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-bjm6s" Jun 08 18:34:48.975599 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:48.975524 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-bjm6s" Jun 08 18:34:49.141137 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:34:49.141105 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf5833ca_6986_401a_a984_d253b6dbc2b2.slice/crio-62d297c8938ca54570c1c22c2a0ab15e8379d068e761dac576f3049b76c2791e WatchSource:0}: Error finding container 62d297c8938ca54570c1c22c2a0ab15e8379d068e761dac576f3049b76c2791e: Status 404 returned error can't find the container with id 62d297c8938ca54570c1c22c2a0ab15e8379d068e761dac576f3049b76c2791e Jun 08 18:34:49.153992 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:49.153949 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-bjm6s"] Jun 08 18:34:49.247384 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:49.247302 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-bjm6s" event={"ID":"af5833ca-6986-401a-a984-d253b6dbc2b2","Type":"ContainerStarted","Data":"62d297c8938ca54570c1c22c2a0ab15e8379d068e761dac576f3049b76c2791e"} Jun 08 18:34:52.269734 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:52.269681 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-bjm6s" event={"ID":"af5833ca-6986-401a-a984-d253b6dbc2b2","Type":"ContainerStarted","Data":"37035ebad3ece08b80cbae4dff54de0e0a8ad4c96d06e6c009c9501f56d791ea"} Jun 08 18:34:52.270221 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:52.269780 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-bjm6s" Jun 08 18:34:52.292078 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:52.292025 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-57f65f65fb-bjm6s" podStartSLOduration=1.6481925099999999 podStartE2EDuration="4.292011835s" totalImagesPullingTime="2.643819325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:34:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:34:52.291431028 +0000 UTC m=+1326.145937840" watchObservedRunningTime="2026-06-08 18:34:52.292011835 +0000 UTC m=+1326.146518630" Jun 08 18:34:54.306224 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.306189 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h"] Jun 08 18:34:54.309646 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.309623 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:54.312420 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.312400 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jun 08 18:34:54.312525 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.312437 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-4dwq6\"" Jun 08 18:34:54.312826 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.312812 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 08 18:34:54.313048 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.313036 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jun 08 18:34:54.313104 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.313036 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jun 08 18:34:54.330886 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.330863 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h"] Jun 08 18:34:54.442755 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.442725 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c1b3d87-99ba-4f72-b35c-ae1650daaa8c-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-x7j8h\" (UID: \"8c1b3d87-99ba-4f72-b35c-ae1650daaa8c\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:54.442924 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.442762 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8c1b3d87-99ba-4f72-b35c-ae1650daaa8c-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-x7j8h\" (UID: \"8c1b3d87-99ba-4f72-b35c-ae1650daaa8c\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:54.442924 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.442780 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8c1b3d87-99ba-4f72-b35c-ae1650daaa8c-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-x7j8h\" (UID: \"8c1b3d87-99ba-4f72-b35c-ae1650daaa8c\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:54.442924 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.442866 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8c1b3d87-99ba-4f72-b35c-ae1650daaa8c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-x7j8h\" (UID: \"8c1b3d87-99ba-4f72-b35c-ae1650daaa8c\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:54.442924 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.442919 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8c1b3d87-99ba-4f72-b35c-ae1650daaa8c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-x7j8h\" (UID: \"8c1b3d87-99ba-4f72-b35c-ae1650daaa8c\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:54.443077 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.442946 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qnl4\" (UniqueName: \"kubernetes.io/projected/8c1b3d87-99ba-4f72-b35c-ae1650daaa8c-kube-api-access-7qnl4\") pod \"istiod-openshift-gateway-75c67f8887-x7j8h\" (UID: \"8c1b3d87-99ba-4f72-b35c-ae1650daaa8c\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:54.443077 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.443011 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c1b3d87-99ba-4f72-b35c-ae1650daaa8c-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-x7j8h\" (UID: \"8c1b3d87-99ba-4f72-b35c-ae1650daaa8c\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:54.544307 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.544264 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c1b3d87-99ba-4f72-b35c-ae1650daaa8c-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-x7j8h\" (UID: \"8c1b3d87-99ba-4f72-b35c-ae1650daaa8c\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:54.544501 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.544320 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8c1b3d87-99ba-4f72-b35c-ae1650daaa8c-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-x7j8h\" (UID: \"8c1b3d87-99ba-4f72-b35c-ae1650daaa8c\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:54.544501 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.544342 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8c1b3d87-99ba-4f72-b35c-ae1650daaa8c-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-x7j8h\" (UID: \"8c1b3d87-99ba-4f72-b35c-ae1650daaa8c\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:54.544501 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.544386 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8c1b3d87-99ba-4f72-b35c-ae1650daaa8c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-x7j8h\" (UID: \"8c1b3d87-99ba-4f72-b35c-ae1650daaa8c\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:54.544501 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.544454 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8c1b3d87-99ba-4f72-b35c-ae1650daaa8c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-x7j8h\" (UID: \"8c1b3d87-99ba-4f72-b35c-ae1650daaa8c\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:54.544501 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.544495 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qnl4\" (UniqueName: \"kubernetes.io/projected/8c1b3d87-99ba-4f72-b35c-ae1650daaa8c-kube-api-access-7qnl4\") pod \"istiod-openshift-gateway-75c67f8887-x7j8h\" (UID: \"8c1b3d87-99ba-4f72-b35c-ae1650daaa8c\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:54.544701 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.544525 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c1b3d87-99ba-4f72-b35c-ae1650daaa8c-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-x7j8h\" (UID: \"8c1b3d87-99ba-4f72-b35c-ae1650daaa8c\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:54.545369 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.545331 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8c1b3d87-99ba-4f72-b35c-ae1650daaa8c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-x7j8h\" (UID: \"8c1b3d87-99ba-4f72-b35c-ae1650daaa8c\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:54.546874 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.546851 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8c1b3d87-99ba-4f72-b35c-ae1650daaa8c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-x7j8h\" (UID: \"8c1b3d87-99ba-4f72-b35c-ae1650daaa8c\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:54.547172 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.547139 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c1b3d87-99ba-4f72-b35c-ae1650daaa8c-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-x7j8h\" (UID: \"8c1b3d87-99ba-4f72-b35c-ae1650daaa8c\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:54.547330 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.547303 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8c1b3d87-99ba-4f72-b35c-ae1650daaa8c-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-x7j8h\" (UID: \"8c1b3d87-99ba-4f72-b35c-ae1650daaa8c\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:54.547522 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.547501 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8c1b3d87-99ba-4f72-b35c-ae1650daaa8c-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-x7j8h\" (UID: \"8c1b3d87-99ba-4f72-b35c-ae1650daaa8c\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:54.554226 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.554202 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c1b3d87-99ba-4f72-b35c-ae1650daaa8c-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-x7j8h\" (UID: \"8c1b3d87-99ba-4f72-b35c-ae1650daaa8c\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:54.554467 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.554443 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qnl4\" (UniqueName: \"kubernetes.io/projected/8c1b3d87-99ba-4f72-b35c-ae1650daaa8c-kube-api-access-7qnl4\") pod \"istiod-openshift-gateway-75c67f8887-x7j8h\" (UID: \"8c1b3d87-99ba-4f72-b35c-ae1650daaa8c\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:54.619268 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.619172 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:54.757111 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:54.757078 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h"] Jun 08 18:34:54.760860 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:34:54.760827 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c1b3d87_99ba_4f72_b35c_ae1650daaa8c.slice/crio-c9e57e209047b9585384a5dc5b7af7ecf8567fd9e0d0083e73af3bd514ca4860 WatchSource:0}: Error finding container c9e57e209047b9585384a5dc5b7af7ecf8567fd9e0d0083e73af3bd514ca4860: Status 404 returned error can't find the container with id c9e57e209047b9585384a5dc5b7af7ecf8567fd9e0d0083e73af3bd514ca4860 Jun 08 18:34:55.283729 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:55.283694 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" event={"ID":"8c1b3d87-99ba-4f72-b35c-ae1650daaa8c","Type":"ContainerStarted","Data":"c9e57e209047b9585384a5dc5b7af7ecf8567fd9e0d0083e73af3bd514ca4860"} Jun 08 18:34:57.229357 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:57.229314 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 08 18:34:57.229657 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:57.229382 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 08 18:34:57.295828 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:57.295795 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" event={"ID":"8c1b3d87-99ba-4f72-b35c-ae1650daaa8c","Type":"ContainerStarted","Data":"1945dd2cf3ed8b4cbd01822fa5d1d0c784da861199ce10afcbdd72350ac4d661"} Jun 08 18:34:57.295936 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:57.295884 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:34:57.324917 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:57.324853 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" podStartSLOduration=0.858816582 podStartE2EDuration="3.324836489s" totalImagesPullingTime="2.466019907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:34:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:34:57.323096904 +0000 UTC m=+1331.177603704" watchObservedRunningTime="2026-06-08 18:34:57.324836489 +0000 UTC m=+1331.179343286" Jun 08 18:34:58.301840 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:58.301804 2572 patch_prober.go:28] interesting pod/istiod-openshift-gateway-75c67f8887-x7j8h container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jun 08 18:34:58.302444 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:34:58.301864 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" podUID="8c1b3d87-99ba-4f72-b35c-ae1650daaa8c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 18:35:01.301480 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:35:01.301442 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-x7j8h" Jun 08 18:35:03.276299 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:35:03.276268 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-bjm6s" Jun 08 18:35:05.258200 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:35:05.258110 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-7fbc7b66c7-vbxq5"] Jun 08 18:35:05.264632 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:35:05.264606 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-7fbc7b66c7-vbxq5" Jun 08 18:35:05.267223 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:35:05.267180 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-dmzsr\"" Jun 08 18:35:05.267455 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:35:05.267438 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 08 18:35:05.272442 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:35:05.272337 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-7fbc7b66c7-vbxq5"] Jun 08 18:35:05.334890 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:35:05.334854 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/30bdecef-135f-434f-ade4-126e684ab92d-redirect-config\") pod \"dashboard-redirect-7fbc7b66c7-vbxq5\" (UID: \"30bdecef-135f-434f-ade4-126e684ab92d\") " pod="opendatahub/dashboard-redirect-7fbc7b66c7-vbxq5" Jun 08 18:35:05.334890 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:35:05.334887 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxq67\" (UniqueName: \"kubernetes.io/projected/30bdecef-135f-434f-ade4-126e684ab92d-kube-api-access-dxq67\") pod \"dashboard-redirect-7fbc7b66c7-vbxq5\" (UID: \"30bdecef-135f-434f-ade4-126e684ab92d\") " pod="opendatahub/dashboard-redirect-7fbc7b66c7-vbxq5" Jun 08 18:35:05.435517 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:35:05.435469 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/30bdecef-135f-434f-ade4-126e684ab92d-redirect-config\") pod \"dashboard-redirect-7fbc7b66c7-vbxq5\" (UID: \"30bdecef-135f-434f-ade4-126e684ab92d\") " pod="opendatahub/dashboard-redirect-7fbc7b66c7-vbxq5" Jun 08 18:35:05.435517 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:35:05.435521 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxq67\" (UniqueName: \"kubernetes.io/projected/30bdecef-135f-434f-ade4-126e684ab92d-kube-api-access-dxq67\") pod \"dashboard-redirect-7fbc7b66c7-vbxq5\" (UID: \"30bdecef-135f-434f-ade4-126e684ab92d\") " pod="opendatahub/dashboard-redirect-7fbc7b66c7-vbxq5" Jun 08 18:35:05.436211 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:35:05.436192 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/30bdecef-135f-434f-ade4-126e684ab92d-redirect-config\") pod \"dashboard-redirect-7fbc7b66c7-vbxq5\" (UID: \"30bdecef-135f-434f-ade4-126e684ab92d\") " pod="opendatahub/dashboard-redirect-7fbc7b66c7-vbxq5" Jun 08 18:35:05.445224 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:35:05.445191 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxq67\" (UniqueName: \"kubernetes.io/projected/30bdecef-135f-434f-ade4-126e684ab92d-kube-api-access-dxq67\") pod \"dashboard-redirect-7fbc7b66c7-vbxq5\" (UID: \"30bdecef-135f-434f-ade4-126e684ab92d\") " pod="opendatahub/dashboard-redirect-7fbc7b66c7-vbxq5" Jun 08 18:35:05.577918 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:35:05.577822 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-7fbc7b66c7-vbxq5" Jun 08 18:35:05.710534 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:35:05.710387 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-7fbc7b66c7-vbxq5"] Jun 08 18:35:05.712668 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:35:05.712639 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30bdecef_135f_434f_ade4_126e684ab92d.slice/crio-120885adf8566404ba16e77f6d76bcd9cbd7cf8f8978ec7f84cdce94a342d720 WatchSource:0}: Error finding container 120885adf8566404ba16e77f6d76bcd9cbd7cf8f8978ec7f84cdce94a342d720: Status 404 returned error can't find the container with id 120885adf8566404ba16e77f6d76bcd9cbd7cf8f8978ec7f84cdce94a342d720 Jun 08 18:35:06.336025 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:35:06.335989 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-7fbc7b66c7-vbxq5" event={"ID":"30bdecef-135f-434f-ade4-126e684ab92d","Type":"ContainerStarted","Data":"120885adf8566404ba16e77f6d76bcd9cbd7cf8f8978ec7f84cdce94a342d720"} Jun 08 18:35:09.351598 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:35:09.351557 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-7fbc7b66c7-vbxq5" event={"ID":"30bdecef-135f-434f-ade4-126e684ab92d","Type":"ContainerStarted","Data":"34fbeaacc07f0a73597d769c22554946d7c13343fce4a793f161b2f5123fd6e5"} Jun 08 18:35:09.369514 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:35:09.369467 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-7fbc7b66c7-vbxq5" podStartSLOduration=1.6503735370000001 podStartE2EDuration="4.369452946s" totalImagesPullingTime="2.719079409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:35:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:35:09.367088615 +0000 UTC m=+1343.221595426" watchObservedRunningTime="2026-06-08 18:35:09.369452946 +0000 UTC m=+1343.223959743" Jun 08 18:35:16.237857 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:35:16.237818 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-69fc866d76-pvcqw" Jun 08 18:35:18.243658 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:35:18.243629 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-z2vmb" Jun 08 18:36:05.773424 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:05.773388 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpdr9"] Jun 08 18:36:05.775739 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:05.775723 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpdr9" Jun 08 18:36:05.778075 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:05.778011 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 08 18:36:05.778470 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:05.778453 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-5j929\"" Jun 08 18:36:05.778691 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:05.778672 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 08 18:36:05.779002 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:05.778989 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 08 18:36:05.789773 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:05.789751 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpdr9"] Jun 08 18:36:05.853273 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:05.853238 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7nkp\" (UniqueName: \"kubernetes.io/projected/479ea4f2-e2ef-4165-b4ec-5f489bab67cf-kube-api-access-p7nkp\") pod \"dns-operator-controller-manager-864dd78c4c-hpdr9\" (UID: \"479ea4f2-e2ef-4165-b4ec-5f489bab67cf\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpdr9" Jun 08 18:36:05.954515 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:05.954486 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7nkp\" (UniqueName: \"kubernetes.io/projected/479ea4f2-e2ef-4165-b4ec-5f489bab67cf-kube-api-access-p7nkp\") pod \"dns-operator-controller-manager-864dd78c4c-hpdr9\" (UID: \"479ea4f2-e2ef-4165-b4ec-5f489bab67cf\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpdr9" Jun 08 18:36:05.963543 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:05.963515 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7nkp\" (UniqueName: \"kubernetes.io/projected/479ea4f2-e2ef-4165-b4ec-5f489bab67cf-kube-api-access-p7nkp\") pod \"dns-operator-controller-manager-864dd78c4c-hpdr9\" (UID: \"479ea4f2-e2ef-4165-b4ec-5f489bab67cf\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpdr9" Jun 08 18:36:06.086055 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:06.085928 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpdr9" Jun 08 18:36:06.217024 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:06.216992 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpdr9"] Jun 08 18:36:06.220889 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:36:06.220861 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod479ea4f2_e2ef_4165_b4ec_5f489bab67cf.slice/crio-590e9682d5019d0d37176a46294608f270b9fd5025f2e283032f112684318f45 WatchSource:0}: Error finding container 590e9682d5019d0d37176a46294608f270b9fd5025f2e283032f112684318f45: Status 404 returned error can't find the container with id 590e9682d5019d0d37176a46294608f270b9fd5025f2e283032f112684318f45 Jun 08 18:36:06.596306 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:06.596275 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpdr9" event={"ID":"479ea4f2-e2ef-4165-b4ec-5f489bab67cf","Type":"ContainerStarted","Data":"590e9682d5019d0d37176a46294608f270b9fd5025f2e283032f112684318f45"} Jun 08 18:36:08.607398 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:08.607360 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpdr9" event={"ID":"479ea4f2-e2ef-4165-b4ec-5f489bab67cf","Type":"ContainerStarted","Data":"5955a30b3252e10b826471ddc60b1616bac3e4a595712c073e3887f9280d2a0a"} Jun 08 18:36:08.607799 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:08.607477 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpdr9" Jun 08 18:36:08.629819 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:08.629763 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpdr9" podStartSLOduration=1.439866362 podStartE2EDuration="3.629744008s" totalImagesPullingTime="2.189877646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:36:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:36:08.628549497 +0000 UTC m=+1402.483056294" watchObservedRunningTime="2026-06-08 18:36:08.629744008 +0000 UTC m=+1402.484250806" Jun 08 18:36:09.950630 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:09.950593 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wflhs"] Jun 08 18:36:09.952852 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:09.952832 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wflhs" Jun 08 18:36:09.957366 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:09.957343 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-42b4f\"" Jun 08 18:36:09.975758 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:09.975727 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wflhs"] Jun 08 18:36:09.991058 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:09.991029 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ktls\" (UniqueName: \"kubernetes.io/projected/c584d751-2229-4b3b-be91-838240db88b5-kube-api-access-6ktls\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wflhs\" (UID: \"c584d751-2229-4b3b-be91-838240db88b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wflhs" Jun 08 18:36:09.991205 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:09.991168 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c584d751-2229-4b3b-be91-838240db88b5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wflhs\" (UID: \"c584d751-2229-4b3b-be91-838240db88b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wflhs" Jun 08 18:36:10.092137 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:10.092095 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ktls\" (UniqueName: \"kubernetes.io/projected/c584d751-2229-4b3b-be91-838240db88b5-kube-api-access-6ktls\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wflhs\" (UID: \"c584d751-2229-4b3b-be91-838240db88b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wflhs" Jun 08 18:36:10.092296 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:10.092166 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c584d751-2229-4b3b-be91-838240db88b5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wflhs\" (UID: \"c584d751-2229-4b3b-be91-838240db88b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wflhs" Jun 08 18:36:10.092534 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:10.092514 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c584d751-2229-4b3b-be91-838240db88b5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wflhs\" (UID: \"c584d751-2229-4b3b-be91-838240db88b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wflhs" Jun 08 18:36:10.103472 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:10.103450 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ktls\" (UniqueName: \"kubernetes.io/projected/c584d751-2229-4b3b-be91-838240db88b5-kube-api-access-6ktls\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wflhs\" (UID: \"c584d751-2229-4b3b-be91-838240db88b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wflhs" Jun 08 18:36:10.265170 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:10.265076 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wflhs" Jun 08 18:36:10.403116 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:10.403088 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wflhs"] Jun 08 18:36:10.405913 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:36:10.405875 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc584d751_2229_4b3b_be91_838240db88b5.slice/crio-513e1078406c23623c525fbc3f9b3200cad8d555ac81bae58af7e9ae689156fd WatchSource:0}: Error finding container 513e1078406c23623c525fbc3f9b3200cad8d555ac81bae58af7e9ae689156fd: Status 404 returned error can't find the container with id 513e1078406c23623c525fbc3f9b3200cad8d555ac81bae58af7e9ae689156fd Jun 08 18:36:10.617040 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:10.616945 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wflhs" event={"ID":"c584d751-2229-4b3b-be91-838240db88b5","Type":"ContainerStarted","Data":"513e1078406c23623c525fbc3f9b3200cad8d555ac81bae58af7e9ae689156fd"} Jun 08 18:36:19.613911 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:19.613876 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hpdr9" Jun 08 18:36:19.667835 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:19.667794 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wflhs" event={"ID":"c584d751-2229-4b3b-be91-838240db88b5","Type":"ContainerStarted","Data":"ed64662b73655611d06c7178d6c98769b3ae309c8b24551c3859f4cf8ee1eaed"} Jun 08 18:36:19.668012 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:19.667897 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wflhs" Jun 08 18:36:19.687348 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:19.687299 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wflhs" podStartSLOduration=2.034048921 podStartE2EDuration="10.687284864s" totalImagesPullingTime="8.653235943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:36:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:36:19.686814698 +0000 UTC m=+1413.541321496" watchObservedRunningTime="2026-06-08 18:36:19.687284864 +0000 UTC m=+1413.541791661" Jun 08 18:36:30.675337 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:30.675297 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wflhs" Jun 08 18:36:32.556769 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.556731 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wflhs"] Jun 08 18:36:32.557193 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.556918 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wflhs" podUID="c584d751-2229-4b3b-be91-838240db88b5" containerName="manager" containerID="cri-o://ed64662b73655611d06c7178d6c98769b3ae309c8b24551c3859f4cf8ee1eaed" gracePeriod=2 Jun 08 18:36:32.570498 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.570461 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wflhs"] Jun 08 18:36:32.573616 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:36:32.573576 2572 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"c584d751-2229-4b3b-be91-838240db88b5\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-wflhs\": pods \"kuadrant-operator-controller-manager-8684c9ff49-wflhs\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wflhs" Jun 08 18:36:32.584017 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.583988 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h"] Jun 08 18:36:32.584381 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.584366 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c584d751-2229-4b3b-be91-838240db88b5" containerName="manager" Jun 08 18:36:32.584423 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.584383 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c584d751-2229-4b3b-be91-838240db88b5" containerName="manager" Jun 08 18:36:32.584480 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.584470 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c584d751-2229-4b3b-be91-838240db88b5" containerName="manager" Jun 08 18:36:32.587759 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.587737 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h" Jun 08 18:36:32.590521 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:36:32.590377 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-wflhs\" is forbidden: User \"system:node:ip-10-0-138-194.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-194.ec2.internal' and this object" podUID="c584d751-2229-4b3b-be91-838240db88b5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wflhs" Jun 08 18:36:32.602524 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.602495 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h"] Jun 08 18:36:32.692770 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.692735 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmw2m\" (UniqueName: \"kubernetes.io/projected/1c029e5c-e68f-4dc8-b452-dd2af93ec04a-kube-api-access-pmw2m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d6k5h\" (UID: \"1c029e5c-e68f-4dc8-b452-dd2af93ec04a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h" Jun 08 18:36:32.692920 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.692788 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1c029e5c-e68f-4dc8-b452-dd2af93ec04a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d6k5h\" (UID: \"1c029e5c-e68f-4dc8-b452-dd2af93ec04a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h" Jun 08 18:36:32.727373 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.727337 2572 generic.go:356] "Generic (PLEG): container finished" podID="c584d751-2229-4b3b-be91-838240db88b5" containerID="ed64662b73655611d06c7178d6c98769b3ae309c8b24551c3859f4cf8ee1eaed" exitCode=0 Jun 08 18:36:32.793425 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.793396 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmw2m\" (UniqueName: \"kubernetes.io/projected/1c029e5c-e68f-4dc8-b452-dd2af93ec04a-kube-api-access-pmw2m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d6k5h\" (UID: \"1c029e5c-e68f-4dc8-b452-dd2af93ec04a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h" Jun 08 18:36:32.793551 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.793442 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1c029e5c-e68f-4dc8-b452-dd2af93ec04a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d6k5h\" (UID: \"1c029e5c-e68f-4dc8-b452-dd2af93ec04a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h" Jun 08 18:36:32.793769 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.793752 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1c029e5c-e68f-4dc8-b452-dd2af93ec04a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d6k5h\" (UID: \"1c029e5c-e68f-4dc8-b452-dd2af93ec04a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h" Jun 08 18:36:32.803213 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.803179 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmw2m\" (UniqueName: \"kubernetes.io/projected/1c029e5c-e68f-4dc8-b452-dd2af93ec04a-kube-api-access-pmw2m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d6k5h\" (UID: \"1c029e5c-e68f-4dc8-b452-dd2af93ec04a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h" Jun 08 18:36:32.805136 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.805117 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wflhs" Jun 08 18:36:32.894417 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.894320 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ktls\" (UniqueName: \"kubernetes.io/projected/c584d751-2229-4b3b-be91-838240db88b5-kube-api-access-6ktls\") pod \"c584d751-2229-4b3b-be91-838240db88b5\" (UID: \"c584d751-2229-4b3b-be91-838240db88b5\") " Jun 08 18:36:32.894417 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.894361 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c584d751-2229-4b3b-be91-838240db88b5-extensions-socket-volume\") pod \"c584d751-2229-4b3b-be91-838240db88b5\" (UID: \"c584d751-2229-4b3b-be91-838240db88b5\") " Jun 08 18:36:32.894879 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.894852 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c584d751-2229-4b3b-be91-838240db88b5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c584d751-2229-4b3b-be91-838240db88b5" (UID: "c584d751-2229-4b3b-be91-838240db88b5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 18:36:32.896585 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.896562 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c584d751-2229-4b3b-be91-838240db88b5-kube-api-access-6ktls" (OuterVolumeSpecName: "kube-api-access-6ktls") pod "c584d751-2229-4b3b-be91-838240db88b5" (UID: "c584d751-2229-4b3b-be91-838240db88b5"). InnerVolumeSpecName "kube-api-access-6ktls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:36:32.944877 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.944840 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h" Jun 08 18:36:32.995576 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.995542 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ktls\" (UniqueName: \"kubernetes.io/projected/c584d751-2229-4b3b-be91-838240db88b5-kube-api-access-6ktls\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:36:32.995696 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:32.995582 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c584d751-2229-4b3b-be91-838240db88b5-extensions-socket-volume\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:36:33.103676 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:33.103642 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h"] Jun 08 18:36:33.108011 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:36:33.107983 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c029e5c_e68f_4dc8_b452_dd2af93ec04a.slice/crio-dc5242388d708b2330c4546561ae01fc56675173c03ca8d44aca3bc53c025ec9 WatchSource:0}: Error finding container dc5242388d708b2330c4546561ae01fc56675173c03ca8d44aca3bc53c025ec9: Status 404 returned error can't find the container with id dc5242388d708b2330c4546561ae01fc56675173c03ca8d44aca3bc53c025ec9 Jun 08 18:36:33.733565 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:33.733532 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wflhs" Jun 08 18:36:33.733565 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:33.733549 2572 scope.go:117] "RemoveContainer" containerID="ed64662b73655611d06c7178d6c98769b3ae309c8b24551c3859f4cf8ee1eaed" Jun 08 18:36:33.735367 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:33.735339 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h" event={"ID":"1c029e5c-e68f-4dc8-b452-dd2af93ec04a","Type":"ContainerStarted","Data":"05f8c883fc304e5ed660a77e4f86f8f2cf948c84dad775202ab49d7f0390ce0e"} Jun 08 18:36:33.735477 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:33.735402 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h" event={"ID":"1c029e5c-e68f-4dc8-b452-dd2af93ec04a","Type":"ContainerStarted","Data":"dc5242388d708b2330c4546561ae01fc56675173c03ca8d44aca3bc53c025ec9"} Jun 08 18:36:33.735477 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:33.735444 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h" Jun 08 18:36:33.758486 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:33.758434 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h" podStartSLOduration=1.758418026 podStartE2EDuration="1.758418026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:36:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:36:33.75753631 +0000 UTC m=+1427.612043107" watchObservedRunningTime="2026-06-08 18:36:33.758418026 +0000 UTC m=+1427.612924824" Jun 08 18:36:34.592555 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:34.592519 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c584d751-2229-4b3b-be91-838240db88b5" path="/var/lib/kubelet/pods/c584d751-2229-4b3b-be91-838240db88b5/volumes" Jun 08 18:36:44.743417 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:36:44.743382 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h" Jun 08 18:37:00.406346 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.406314 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h"] Jun 08 18:37:00.406871 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.406559 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h" podUID="1c029e5c-e68f-4dc8-b452-dd2af93ec04a" containerName="manager" containerID="cri-o://05f8c883fc304e5ed660a77e4f86f8f2cf948c84dad775202ab49d7f0390ce0e" gracePeriod=10 Jun 08 18:37:00.620104 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.620066 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7"] Jun 08 18:37:00.622544 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.622523 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7" Jun 08 18:37:00.641491 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.641464 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7"] Jun 08 18:37:00.661893 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.661839 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h" Jun 08 18:37:00.730364 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.730335 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1c029e5c-e68f-4dc8-b452-dd2af93ec04a-extensions-socket-volume\") pod \"1c029e5c-e68f-4dc8-b452-dd2af93ec04a\" (UID: \"1c029e5c-e68f-4dc8-b452-dd2af93ec04a\") " Jun 08 18:37:00.730528 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.730451 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmw2m\" (UniqueName: \"kubernetes.io/projected/1c029e5c-e68f-4dc8-b452-dd2af93ec04a-kube-api-access-pmw2m\") pod \"1c029e5c-e68f-4dc8-b452-dd2af93ec04a\" (UID: \"1c029e5c-e68f-4dc8-b452-dd2af93ec04a\") " Jun 08 18:37:00.730593 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.730574 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgqjz\" (UniqueName: \"kubernetes.io/projected/c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2-kube-api-access-dgqjz\") pod \"kuadrant-operator-controller-manager-86878c769f-f9vz7\" (UID: \"c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7" Jun 08 18:37:00.730654 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.730636 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-f9vz7\" (UID: \"c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7" Jun 08 18:37:00.730792 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.730774 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c029e5c-e68f-4dc8-b452-dd2af93ec04a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1c029e5c-e68f-4dc8-b452-dd2af93ec04a" (UID: "1c029e5c-e68f-4dc8-b452-dd2af93ec04a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 18:37:00.732490 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.732456 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c029e5c-e68f-4dc8-b452-dd2af93ec04a-kube-api-access-pmw2m" (OuterVolumeSpecName: "kube-api-access-pmw2m") pod "1c029e5c-e68f-4dc8-b452-dd2af93ec04a" (UID: "1c029e5c-e68f-4dc8-b452-dd2af93ec04a"). InnerVolumeSpecName "kube-api-access-pmw2m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:37:00.832151 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.832097 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgqjz\" (UniqueName: \"kubernetes.io/projected/c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2-kube-api-access-dgqjz\") pod \"kuadrant-operator-controller-manager-86878c769f-f9vz7\" (UID: \"c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7" Jun 08 18:37:00.832323 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.832161 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-f9vz7\" (UID: \"c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7" Jun 08 18:37:00.832323 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.832198 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmw2m\" (UniqueName: \"kubernetes.io/projected/1c029e5c-e68f-4dc8-b452-dd2af93ec04a-kube-api-access-pmw2m\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:37:00.832323 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.832209 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1c029e5c-e68f-4dc8-b452-dd2af93ec04a-extensions-socket-volume\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:37:00.832552 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.832536 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-f9vz7\" (UID: \"c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7" Jun 08 18:37:00.845182 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.845148 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgqjz\" (UniqueName: \"kubernetes.io/projected/c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2-kube-api-access-dgqjz\") pod \"kuadrant-operator-controller-manager-86878c769f-f9vz7\" (UID: \"c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7" Jun 08 18:37:00.857909 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.857877 2572 generic.go:356] "Generic (PLEG): container finished" podID="1c029e5c-e68f-4dc8-b452-dd2af93ec04a" containerID="05f8c883fc304e5ed660a77e4f86f8f2cf948c84dad775202ab49d7f0390ce0e" exitCode=0 Jun 08 18:37:00.858056 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.857924 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h" Jun 08 18:37:00.858056 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.857934 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h" event={"ID":"1c029e5c-e68f-4dc8-b452-dd2af93ec04a","Type":"ContainerDied","Data":"05f8c883fc304e5ed660a77e4f86f8f2cf948c84dad775202ab49d7f0390ce0e"} Jun 08 18:37:00.858056 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.858001 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h" event={"ID":"1c029e5c-e68f-4dc8-b452-dd2af93ec04a","Type":"ContainerDied","Data":"dc5242388d708b2330c4546561ae01fc56675173c03ca8d44aca3bc53c025ec9"} Jun 08 18:37:00.858056 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.858026 2572 scope.go:117] "RemoveContainer" containerID="05f8c883fc304e5ed660a77e4f86f8f2cf948c84dad775202ab49d7f0390ce0e" Jun 08 18:37:00.866312 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.866294 2572 scope.go:117] "RemoveContainer" containerID="05f8c883fc304e5ed660a77e4f86f8f2cf948c84dad775202ab49d7f0390ce0e" Jun 08 18:37:00.866569 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:37:00.866550 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05f8c883fc304e5ed660a77e4f86f8f2cf948c84dad775202ab49d7f0390ce0e\": container with ID starting with 05f8c883fc304e5ed660a77e4f86f8f2cf948c84dad775202ab49d7f0390ce0e not found: ID does not exist" containerID="05f8c883fc304e5ed660a77e4f86f8f2cf948c84dad775202ab49d7f0390ce0e" Jun 08 18:37:00.866633 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.866577 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05f8c883fc304e5ed660a77e4f86f8f2cf948c84dad775202ab49d7f0390ce0e"} err="failed to get container status \"05f8c883fc304e5ed660a77e4f86f8f2cf948c84dad775202ab49d7f0390ce0e\": rpc error: code = NotFound desc = could not find container \"05f8c883fc304e5ed660a77e4f86f8f2cf948c84dad775202ab49d7f0390ce0e\": container with ID starting with 05f8c883fc304e5ed660a77e4f86f8f2cf948c84dad775202ab49d7f0390ce0e not found: ID does not exist" Jun 08 18:37:00.883268 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.883239 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h"] Jun 08 18:37:00.890014 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.889990 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6k5h"] Jun 08 18:37:00.933917 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:00.933847 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7" Jun 08 18:37:01.068046 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:01.068007 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7"] Jun 08 18:37:01.071751 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:37:01.071718 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8c4b40d_ca2f_4d1e_a66f_5d97e732b6c2.slice/crio-3962eee0e9e43b16184ce5e02006a8ba4fc91b88150045a467e757dbed0b73da WatchSource:0}: Error finding container 3962eee0e9e43b16184ce5e02006a8ba4fc91b88150045a467e757dbed0b73da: Status 404 returned error can't find the container with id 3962eee0e9e43b16184ce5e02006a8ba4fc91b88150045a467e757dbed0b73da Jun 08 18:37:01.864728 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:01.864692 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7" event={"ID":"c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2","Type":"ContainerStarted","Data":"673778bad28f3d9f2d416f212b352117cc6931a34a4c7526a6f00834ffce8dd5"} Jun 08 18:37:01.865164 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:01.864740 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7" event={"ID":"c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2","Type":"ContainerStarted","Data":"3962eee0e9e43b16184ce5e02006a8ba4fc91b88150045a467e757dbed0b73da"} Jun 08 18:37:01.865164 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:01.864817 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7" Jun 08 18:37:01.898198 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:01.898143 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7" podStartSLOduration=1.8981262829999999 podStartE2EDuration="1.898126283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:37:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:37:01.891536397 +0000 UTC m=+1455.746043195" watchObservedRunningTime="2026-06-08 18:37:01.898126283 +0000 UTC m=+1455.752633080" Jun 08 18:37:02.598036 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:02.598003 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c029e5c-e68f-4dc8-b452-dd2af93ec04a" path="/var/lib/kubelet/pods/1c029e5c-e68f-4dc8-b452-dd2af93ec04a/volumes" Jun 08 18:37:12.871707 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:12.871673 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7" Jun 08 18:37:20.716514 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:20.716476 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jjrw2"] Jun 08 18:37:20.716937 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:20.716852 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c029e5c-e68f-4dc8-b452-dd2af93ec04a" containerName="manager" Jun 08 18:37:20.716937 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:20.716864 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c029e5c-e68f-4dc8-b452-dd2af93ec04a" containerName="manager" Jun 08 18:37:20.716937 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:20.716924 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c029e5c-e68f-4dc8-b452-dd2af93ec04a" containerName="manager" Jun 08 18:37:20.719772 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:20.719755 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjrw2" Jun 08 18:37:20.722892 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:20.722867 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2jxxd\"" Jun 08 18:37:20.723053 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:20.722923 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 08 18:37:20.738689 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:20.738664 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jjrw2"] Jun 08 18:37:20.805380 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:20.805341 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fd81d516-bdc4-474b-ab79-fc3b1b8d4808-config-file\") pod \"limitador-limitador-64c94fd64f-jjrw2\" (UID: \"fd81d516-bdc4-474b-ab79-fc3b1b8d4808\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jjrw2" Jun 08 18:37:20.805539 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:20.805478 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crlvt\" (UniqueName: \"kubernetes.io/projected/fd81d516-bdc4-474b-ab79-fc3b1b8d4808-kube-api-access-crlvt\") pod \"limitador-limitador-64c94fd64f-jjrw2\" (UID: \"fd81d516-bdc4-474b-ab79-fc3b1b8d4808\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jjrw2" Jun 08 18:37:20.812667 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:20.812633 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jjrw2"] Jun 08 18:37:20.906428 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:20.906388 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crlvt\" (UniqueName: \"kubernetes.io/projected/fd81d516-bdc4-474b-ab79-fc3b1b8d4808-kube-api-access-crlvt\") pod \"limitador-limitador-64c94fd64f-jjrw2\" (UID: \"fd81d516-bdc4-474b-ab79-fc3b1b8d4808\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jjrw2" Jun 08 18:37:20.906642 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:20.906472 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fd81d516-bdc4-474b-ab79-fc3b1b8d4808-config-file\") pod \"limitador-limitador-64c94fd64f-jjrw2\" (UID: \"fd81d516-bdc4-474b-ab79-fc3b1b8d4808\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jjrw2" Jun 08 18:37:20.907270 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:20.907243 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fd81d516-bdc4-474b-ab79-fc3b1b8d4808-config-file\") pod \"limitador-limitador-64c94fd64f-jjrw2\" (UID: \"fd81d516-bdc4-474b-ab79-fc3b1b8d4808\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jjrw2" Jun 08 18:37:20.914335 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:20.914309 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crlvt\" (UniqueName: \"kubernetes.io/projected/fd81d516-bdc4-474b-ab79-fc3b1b8d4808-kube-api-access-crlvt\") pod \"limitador-limitador-64c94fd64f-jjrw2\" (UID: \"fd81d516-bdc4-474b-ab79-fc3b1b8d4808\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jjrw2" Jun 08 18:37:21.030678 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:21.030584 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjrw2" Jun 08 18:37:21.166856 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:21.166822 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jjrw2"] Jun 08 18:37:21.171021 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:37:21.170992 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd81d516_bdc4_474b_ab79_fc3b1b8d4808.slice/crio-4c8816b5e9d090564b9df8e57f606df9ab3f9660264d7a2a73b7527693bbac4d WatchSource:0}: Error finding container 4c8816b5e9d090564b9df8e57f606df9ab3f9660264d7a2a73b7527693bbac4d: Status 404 returned error can't find the container with id 4c8816b5e9d090564b9df8e57f606df9ab3f9660264d7a2a73b7527693bbac4d Jun 08 18:37:21.976100 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:21.976022 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjrw2" event={"ID":"fd81d516-bdc4-474b-ab79-fc3b1b8d4808","Type":"ContainerStarted","Data":"4c8816b5e9d090564b9df8e57f606df9ab3f9660264d7a2a73b7527693bbac4d"} Jun 08 18:37:23.986715 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:23.986678 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjrw2" event={"ID":"fd81d516-bdc4-474b-ab79-fc3b1b8d4808","Type":"ContainerStarted","Data":"51970778911a59fe10ff824a735b982a913f65d86d0e041265edf57f39b8afaa"} Jun 08 18:37:23.987141 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:23.986816 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjrw2" Jun 08 18:37:24.002652 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:24.002601 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjrw2" podStartSLOduration=1.412665648 podStartE2EDuration="4.00258147s" totalImagesPullingTime="2.589915822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:37:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:37:24.002184767 +0000 UTC m=+1477.856691566" watchObservedRunningTime="2026-06-08 18:37:24.00258147 +0000 UTC m=+1477.857088271" Jun 08 18:37:34.992491 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:34.992460 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjrw2" Jun 08 18:37:35.559050 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:35.559012 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jjrw2"] Jun 08 18:37:35.559244 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:35.559205 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjrw2" podUID="fd81d516-bdc4-474b-ab79-fc3b1b8d4808" containerName="limitador" containerID="cri-o://51970778911a59fe10ff824a735b982a913f65d86d0e041265edf57f39b8afaa" gracePeriod=30 Jun 08 18:37:36.042045 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:36.042013 2572 generic.go:356] "Generic (PLEG): container finished" podID="fd81d516-bdc4-474b-ab79-fc3b1b8d4808" containerID="51970778911a59fe10ff824a735b982a913f65d86d0e041265edf57f39b8afaa" exitCode=0 Jun 08 18:37:36.042389 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:36.042057 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjrw2" event={"ID":"fd81d516-bdc4-474b-ab79-fc3b1b8d4808","Type":"ContainerDied","Data":"51970778911a59fe10ff824a735b982a913f65d86d0e041265edf57f39b8afaa"} Jun 08 18:37:36.103029 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:36.103002 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjrw2" Jun 08 18:37:36.248390 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:36.248355 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fd81d516-bdc4-474b-ab79-fc3b1b8d4808-config-file\") pod \"fd81d516-bdc4-474b-ab79-fc3b1b8d4808\" (UID: \"fd81d516-bdc4-474b-ab79-fc3b1b8d4808\") " Jun 08 18:37:36.248552 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:36.248407 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crlvt\" (UniqueName: \"kubernetes.io/projected/fd81d516-bdc4-474b-ab79-fc3b1b8d4808-kube-api-access-crlvt\") pod \"fd81d516-bdc4-474b-ab79-fc3b1b8d4808\" (UID: \"fd81d516-bdc4-474b-ab79-fc3b1b8d4808\") " Jun 08 18:37:36.248723 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:36.248698 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd81d516-bdc4-474b-ab79-fc3b1b8d4808-config-file" (OuterVolumeSpecName: "config-file") pod "fd81d516-bdc4-474b-ab79-fc3b1b8d4808" (UID: "fd81d516-bdc4-474b-ab79-fc3b1b8d4808"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 18:37:36.250439 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:36.250408 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd81d516-bdc4-474b-ab79-fc3b1b8d4808-kube-api-access-crlvt" (OuterVolumeSpecName: "kube-api-access-crlvt") pod "fd81d516-bdc4-474b-ab79-fc3b1b8d4808" (UID: "fd81d516-bdc4-474b-ab79-fc3b1b8d4808"). InnerVolumeSpecName "kube-api-access-crlvt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:37:36.354225 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:36.349663 2572 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fd81d516-bdc4-474b-ab79-fc3b1b8d4808-config-file\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:37:36.354225 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:36.349699 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crlvt\" (UniqueName: \"kubernetes.io/projected/fd81d516-bdc4-474b-ab79-fc3b1b8d4808-kube-api-access-crlvt\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:37:36.839704 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:36.839666 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-7h5bg"] Jun 08 18:37:36.840223 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:36.840202 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd81d516-bdc4-474b-ab79-fc3b1b8d4808" containerName="limitador" Jun 08 18:37:36.840278 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:36.840228 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd81d516-bdc4-474b-ab79-fc3b1b8d4808" containerName="limitador" Jun 08 18:37:36.840362 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:36.840349 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd81d516-bdc4-474b-ab79-fc3b1b8d4808" containerName="limitador" Jun 08 18:37:36.843783 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:36.843761 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-7h5bg" Jun 08 18:37:36.845939 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:36.845914 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jun 08 18:37:36.849542 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:36.849479 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-7h5bg"] Jun 08 18:37:36.855011 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:36.854989 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2279cfbc-3808-4891-ad48-f5331edea3f3-data\") pod \"postgres-69fb4594cf-7h5bg\" (UID: \"2279cfbc-3808-4891-ad48-f5331edea3f3\") " pod="opendatahub/postgres-69fb4594cf-7h5bg" Jun 08 18:37:36.855124 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:36.855067 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7h77\" (UniqueName: \"kubernetes.io/projected/2279cfbc-3808-4891-ad48-f5331edea3f3-kube-api-access-r7h77\") pod \"postgres-69fb4594cf-7h5bg\" (UID: \"2279cfbc-3808-4891-ad48-f5331edea3f3\") " pod="opendatahub/postgres-69fb4594cf-7h5bg" Jun 08 18:37:36.955722 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:36.955681 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2279cfbc-3808-4891-ad48-f5331edea3f3-data\") pod \"postgres-69fb4594cf-7h5bg\" (UID: \"2279cfbc-3808-4891-ad48-f5331edea3f3\") " pod="opendatahub/postgres-69fb4594cf-7h5bg" Jun 08 18:37:36.955901 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:36.955762 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7h77\" (UniqueName: \"kubernetes.io/projected/2279cfbc-3808-4891-ad48-f5331edea3f3-kube-api-access-r7h77\") pod \"postgres-69fb4594cf-7h5bg\" (UID: \"2279cfbc-3808-4891-ad48-f5331edea3f3\") " pod="opendatahub/postgres-69fb4594cf-7h5bg" Jun 08 18:37:36.956139 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:36.956120 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2279cfbc-3808-4891-ad48-f5331edea3f3-data\") pod \"postgres-69fb4594cf-7h5bg\" (UID: \"2279cfbc-3808-4891-ad48-f5331edea3f3\") " pod="opendatahub/postgres-69fb4594cf-7h5bg" Jun 08 18:37:36.966609 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:36.966579 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7h77\" (UniqueName: \"kubernetes.io/projected/2279cfbc-3808-4891-ad48-f5331edea3f3-kube-api-access-r7h77\") pod \"postgres-69fb4594cf-7h5bg\" (UID: \"2279cfbc-3808-4891-ad48-f5331edea3f3\") " pod="opendatahub/postgres-69fb4594cf-7h5bg" Jun 08 18:37:37.051625 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:37.051579 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjrw2" event={"ID":"fd81d516-bdc4-474b-ab79-fc3b1b8d4808","Type":"ContainerDied","Data":"4c8816b5e9d090564b9df8e57f606df9ab3f9660264d7a2a73b7527693bbac4d"} Jun 08 18:37:37.052079 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:37.051656 2572 scope.go:117] "RemoveContainer" containerID="51970778911a59fe10ff824a735b982a913f65d86d0e041265edf57f39b8afaa" Jun 08 18:37:37.052079 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:37.051837 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jjrw2" Jun 08 18:37:37.074261 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:37.074232 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jjrw2"] Jun 08 18:37:37.077436 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:37.077410 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jjrw2"] Jun 08 18:37:37.155874 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:37.155844 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-7h5bg" Jun 08 18:37:37.293218 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:37.293190 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-7h5bg"] Jun 08 18:37:37.295780 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:37:37.295752 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2279cfbc_3808_4891_ad48_f5331edea3f3.slice/crio-84a8a0c66e3ac14c2f0dfdcd28d8ed7b56e18daff1f123f55fa6a00782fefb9b WatchSource:0}: Error finding container 84a8a0c66e3ac14c2f0dfdcd28d8ed7b56e18daff1f123f55fa6a00782fefb9b: Status 404 returned error can't find the container with id 84a8a0c66e3ac14c2f0dfdcd28d8ed7b56e18daff1f123f55fa6a00782fefb9b Jun 08 18:37:38.058133 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:38.058100 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-7h5bg" event={"ID":"2279cfbc-3808-4891-ad48-f5331edea3f3","Type":"ContainerStarted","Data":"84a8a0c66e3ac14c2f0dfdcd28d8ed7b56e18daff1f123f55fa6a00782fefb9b"} Jun 08 18:37:38.591845 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:38.591812 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd81d516-bdc4-474b-ab79-fc3b1b8d4808" path="/var/lib/kubelet/pods/fd81d516-bdc4-474b-ab79-fc3b1b8d4808/volumes" Jun 08 18:37:41.075028 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:41.074991 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-7h5bg" event={"ID":"2279cfbc-3808-4891-ad48-f5331edea3f3","Type":"ContainerStarted","Data":"3fad98b92679bba20415288b9c75824cc99f08977b7a5664ad8a16630b56ef76"} Jun 08 18:37:41.075502 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:41.075081 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-7h5bg" Jun 08 18:37:41.092573 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:41.092523 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-7h5bg" podStartSLOduration=2.109332388 podStartE2EDuration="5.092509978s" totalImagesPullingTime="2.98317759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:37:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:37:41.090927647 +0000 UTC m=+1494.945434442" watchObservedRunningTime="2026-06-08 18:37:41.092509978 +0000 UTC m=+1494.947016776" Jun 08 18:37:46.634674 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:46.634647 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 18:37:46.636451 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:46.636432 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 18:37:47.112342 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:37:47.112314 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-7h5bg" Jun 08 18:38:51.787832 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:51.787794 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6dff99846d-g5c4s"] Jun 08 18:38:51.791324 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:51.791306 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dff99846d-g5c4s" Jun 08 18:38:51.796573 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:51.796500 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-sg4jj\"" Jun 08 18:38:51.796573 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:51.796515 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 08 18:38:51.796773 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:51.796529 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 08 18:38:51.808677 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:51.808653 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6dff99846d-g5c4s"] Jun 08 18:38:51.908482 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:51.908450 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c-webhook-cert\") pod \"maas-controller-6dff99846d-g5c4s\" (UID: \"a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c\") " pod="opendatahub/maas-controller-6dff99846d-g5c4s" Jun 08 18:38:51.908482 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:51.908483 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b26f\" (UniqueName: \"kubernetes.io/projected/a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c-kube-api-access-2b26f\") pod \"maas-controller-6dff99846d-g5c4s\" (UID: \"a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c\") " pod="opendatahub/maas-controller-6dff99846d-g5c4s" Jun 08 18:38:52.009536 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:52.009497 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c-webhook-cert\") pod \"maas-controller-6dff99846d-g5c4s\" (UID: \"a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c\") " pod="opendatahub/maas-controller-6dff99846d-g5c4s" Jun 08 18:38:52.009536 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:52.009538 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b26f\" (UniqueName: \"kubernetes.io/projected/a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c-kube-api-access-2b26f\") pod \"maas-controller-6dff99846d-g5c4s\" (UID: \"a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c\") " pod="opendatahub/maas-controller-6dff99846d-g5c4s" Jun 08 18:38:52.011884 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:52.011852 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c-webhook-cert\") pod \"maas-controller-6dff99846d-g5c4s\" (UID: \"a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c\") " pod="opendatahub/maas-controller-6dff99846d-g5c4s" Jun 08 18:38:52.030370 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:52.030343 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b26f\" (UniqueName: \"kubernetes.io/projected/a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c-kube-api-access-2b26f\") pod \"maas-controller-6dff99846d-g5c4s\" (UID: \"a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c\") " pod="opendatahub/maas-controller-6dff99846d-g5c4s" Jun 08 18:38:52.054369 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:52.054310 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-548c5ccc8f-9ft5f"] Jun 08 18:38:52.057634 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:52.057619 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-548c5ccc8f-9ft5f" Jun 08 18:38:52.085904 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:52.085876 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-548c5ccc8f-9ft5f"] Jun 08 18:38:52.101884 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:52.101859 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dff99846d-g5c4s" Jun 08 18:38:52.110241 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:52.110215 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzkrc\" (UniqueName: \"kubernetes.io/projected/919ee8aa-7105-4658-a664-2ee87a9ebc76-kube-api-access-fzkrc\") pod \"maas-controller-548c5ccc8f-9ft5f\" (UID: \"919ee8aa-7105-4658-a664-2ee87a9ebc76\") " pod="opendatahub/maas-controller-548c5ccc8f-9ft5f" Jun 08 18:38:52.110361 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:52.110258 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/919ee8aa-7105-4658-a664-2ee87a9ebc76-webhook-cert\") pod \"maas-controller-548c5ccc8f-9ft5f\" (UID: \"919ee8aa-7105-4658-a664-2ee87a9ebc76\") " pod="opendatahub/maas-controller-548c5ccc8f-9ft5f" Jun 08 18:38:52.211720 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:52.211677 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzkrc\" (UniqueName: \"kubernetes.io/projected/919ee8aa-7105-4658-a664-2ee87a9ebc76-kube-api-access-fzkrc\") pod \"maas-controller-548c5ccc8f-9ft5f\" (UID: \"919ee8aa-7105-4658-a664-2ee87a9ebc76\") " pod="opendatahub/maas-controller-548c5ccc8f-9ft5f" Jun 08 18:38:52.211882 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:52.211846 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/919ee8aa-7105-4658-a664-2ee87a9ebc76-webhook-cert\") pod \"maas-controller-548c5ccc8f-9ft5f\" (UID: \"919ee8aa-7105-4658-a664-2ee87a9ebc76\") " pod="opendatahub/maas-controller-548c5ccc8f-9ft5f" Jun 08 18:38:52.214446 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:52.214417 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/919ee8aa-7105-4658-a664-2ee87a9ebc76-webhook-cert\") pod \"maas-controller-548c5ccc8f-9ft5f\" (UID: \"919ee8aa-7105-4658-a664-2ee87a9ebc76\") " pod="opendatahub/maas-controller-548c5ccc8f-9ft5f" Jun 08 18:38:52.219577 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:52.219552 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzkrc\" (UniqueName: \"kubernetes.io/projected/919ee8aa-7105-4658-a664-2ee87a9ebc76-kube-api-access-fzkrc\") pod \"maas-controller-548c5ccc8f-9ft5f\" (UID: \"919ee8aa-7105-4658-a664-2ee87a9ebc76\") " pod="opendatahub/maas-controller-548c5ccc8f-9ft5f" Jun 08 18:38:52.239949 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:52.239927 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6dff99846d-g5c4s"] Jun 08 18:38:52.241638 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:38:52.241613 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda39c3a8a_47c2_46a6_8a34_53d8d6d7b04c.slice/crio-413def4e25260d9a92d8f15beb4dc82e3f5ec5697e1af3ade93950af722cb403 WatchSource:0}: Error finding container 413def4e25260d9a92d8f15beb4dc82e3f5ec5697e1af3ade93950af722cb403: Status 404 returned error can't find the container with id 413def4e25260d9a92d8f15beb4dc82e3f5ec5697e1af3ade93950af722cb403 Jun 08 18:38:52.367755 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:52.367666 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-548c5ccc8f-9ft5f" Jun 08 18:38:52.400285 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:52.400250 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dff99846d-g5c4s" event={"ID":"a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c","Type":"ContainerStarted","Data":"413def4e25260d9a92d8f15beb4dc82e3f5ec5697e1af3ade93950af722cb403"} Jun 08 18:38:52.504637 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:52.504605 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-548c5ccc8f-9ft5f"] Jun 08 18:38:52.507854 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:38:52.507833 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod919ee8aa_7105_4658_a664_2ee87a9ebc76.slice/crio-13a96e13621c8d76f55e15219c8a5e90ad4c2a049d8461be9d917b5ca01c9116 WatchSource:0}: Error finding container 13a96e13621c8d76f55e15219c8a5e90ad4c2a049d8461be9d917b5ca01c9116: Status 404 returned error can't find the container with id 13a96e13621c8d76f55e15219c8a5e90ad4c2a049d8461be9d917b5ca01c9116 Jun 08 18:38:53.410814 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:53.410724 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-548c5ccc8f-9ft5f" event={"ID":"919ee8aa-7105-4658-a664-2ee87a9ebc76","Type":"ContainerStarted","Data":"13a96e13621c8d76f55e15219c8a5e90ad4c2a049d8461be9d917b5ca01c9116"} Jun 08 18:38:55.422454 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:55.422412 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dff99846d-g5c4s" event={"ID":"a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c","Type":"ContainerStarted","Data":"6ce2b4b717cda96f98b1ed008e7753053088890fef2586ba231eb7cde6bcdb6a"} Jun 08 18:38:55.422895 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:55.422616 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6dff99846d-g5c4s" Jun 08 18:38:55.424216 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:55.424194 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-548c5ccc8f-9ft5f" event={"ID":"919ee8aa-7105-4658-a664-2ee87a9ebc76","Type":"ContainerStarted","Data":"643d60b57aaf1d3dfba0117eb135cf097430018fc4f1906956e1f4e6cd1a1597"} Jun 08 18:38:55.424324 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:55.424238 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-548c5ccc8f-9ft5f" Jun 08 18:38:55.454174 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:55.454128 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6dff99846d-g5c4s" podStartSLOduration=1.794671707 podStartE2EDuration="4.454112592s" totalImagesPullingTime="2.659440885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:38:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:38:55.453597566 +0000 UTC m=+1569.308104365" watchObservedRunningTime="2026-06-08 18:38:55.454112592 +0000 UTC m=+1569.308619389" Jun 08 18:38:55.491529 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:38:55.491476 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-548c5ccc8f-9ft5f" podStartSLOduration=1.091540407 podStartE2EDuration="3.491458023s" totalImagesPullingTime="2.399917616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:38:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:38:55.489498061 +0000 UTC m=+1569.344004884" watchObservedRunningTime="2026-06-08 18:38:55.491458023 +0000 UTC m=+1569.345964862" Jun 08 18:39:06.434594 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:06.434557 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-548c5ccc8f-9ft5f" Jun 08 18:39:06.435004 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:06.434818 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6dff99846d-g5c4s" Jun 08 18:39:06.512355 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:06.512326 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6dff99846d-g5c4s"] Jun 08 18:39:06.512537 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:06.512500 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6dff99846d-g5c4s" podUID="a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c" containerName="manager" containerID="cri-o://6ce2b4b717cda96f98b1ed008e7753053088890fef2586ba231eb7cde6bcdb6a" gracePeriod=30 Jun 08 18:39:06.754347 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:06.754322 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dff99846d-g5c4s" Jun 08 18:39:06.850645 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:06.850605 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c-webhook-cert\") pod \"a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c\" (UID: \"a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c\") " Jun 08 18:39:06.850832 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:06.850802 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2b26f\" (UniqueName: \"kubernetes.io/projected/a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c-kube-api-access-2b26f\") pod \"a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c\" (UID: \"a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c\") " Jun 08 18:39:06.852881 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:06.852847 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c-kube-api-access-2b26f" (OuterVolumeSpecName: "kube-api-access-2b26f") pod "a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c" (UID: "a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c"). InnerVolumeSpecName "kube-api-access-2b26f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:39:06.852881 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:06.852873 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c" (UID: "a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:39:06.951497 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:06.951464 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2b26f\" (UniqueName: \"kubernetes.io/projected/a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c-kube-api-access-2b26f\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:39:06.951497 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:06.951495 2572 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c-webhook-cert\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:39:07.480940 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:07.480905 2572 generic.go:356] "Generic (PLEG): container finished" podID="a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c" containerID="6ce2b4b717cda96f98b1ed008e7753053088890fef2586ba231eb7cde6bcdb6a" exitCode=0 Jun 08 18:39:07.481372 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:07.480954 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dff99846d-g5c4s" Jun 08 18:39:07.481372 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:07.480989 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dff99846d-g5c4s" event={"ID":"a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c","Type":"ContainerDied","Data":"6ce2b4b717cda96f98b1ed008e7753053088890fef2586ba231eb7cde6bcdb6a"} Jun 08 18:39:07.481372 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:07.481049 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dff99846d-g5c4s" event={"ID":"a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c","Type":"ContainerDied","Data":"413def4e25260d9a92d8f15beb4dc82e3f5ec5697e1af3ade93950af722cb403"} Jun 08 18:39:07.481372 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:07.481081 2572 scope.go:117] "RemoveContainer" containerID="6ce2b4b717cda96f98b1ed008e7753053088890fef2586ba231eb7cde6bcdb6a" Jun 08 18:39:07.489948 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:07.489929 2572 scope.go:117] "RemoveContainer" containerID="6ce2b4b717cda96f98b1ed008e7753053088890fef2586ba231eb7cde6bcdb6a" Jun 08 18:39:07.490220 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:39:07.490201 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ce2b4b717cda96f98b1ed008e7753053088890fef2586ba231eb7cde6bcdb6a\": container with ID starting with 6ce2b4b717cda96f98b1ed008e7753053088890fef2586ba231eb7cde6bcdb6a not found: ID does not exist" containerID="6ce2b4b717cda96f98b1ed008e7753053088890fef2586ba231eb7cde6bcdb6a" Jun 08 18:39:07.490265 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:07.490231 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ce2b4b717cda96f98b1ed008e7753053088890fef2586ba231eb7cde6bcdb6a"} err="failed to get container status \"6ce2b4b717cda96f98b1ed008e7753053088890fef2586ba231eb7cde6bcdb6a\": rpc error: code = NotFound desc = could not find container \"6ce2b4b717cda96f98b1ed008e7753053088890fef2586ba231eb7cde6bcdb6a\": container with ID starting with 6ce2b4b717cda96f98b1ed008e7753053088890fef2586ba231eb7cde6bcdb6a not found: ID does not exist" Jun 08 18:39:07.503476 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:07.503450 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6dff99846d-g5c4s"] Jun 08 18:39:07.507579 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:07.507561 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6dff99846d-g5c4s"] Jun 08 18:39:08.592216 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:08.592184 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c" path="/var/lib/kubelet/pods/a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c/volumes" Jun 08 18:39:26.406216 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:26.406174 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-859d5d9c66-x7h8t"] Jun 08 18:39:26.408731 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:26.406675 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c" containerName="manager" Jun 08 18:39:26.408731 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:26.406691 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c" containerName="manager" Jun 08 18:39:26.408731 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:26.406761 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a39c3a8a-47c2-46a6-8a34-53d8d6d7b04c" containerName="manager" Jun 08 18:39:26.409581 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:26.409562 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-859d5d9c66-x7h8t" Jun 08 18:39:26.416001 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:26.415952 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 08 18:39:26.432786 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:26.432756 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-859d5d9c66-x7h8t"] Jun 08 18:39:26.518535 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:26.518493 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69de6a4a-777e-4e70-9a9c-fe7198f9ca9c-maas-api-tls\") pod \"maas-api-859d5d9c66-x7h8t\" (UID: \"69de6a4a-777e-4e70-9a9c-fe7198f9ca9c\") " pod="opendatahub/maas-api-859d5d9c66-x7h8t" Jun 08 18:39:26.518535 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:26.518533 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg8q6\" (UniqueName: \"kubernetes.io/projected/69de6a4a-777e-4e70-9a9c-fe7198f9ca9c-kube-api-access-wg8q6\") pod \"maas-api-859d5d9c66-x7h8t\" (UID: \"69de6a4a-777e-4e70-9a9c-fe7198f9ca9c\") " pod="opendatahub/maas-api-859d5d9c66-x7h8t" Jun 08 18:39:26.619106 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:26.619068 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69de6a4a-777e-4e70-9a9c-fe7198f9ca9c-maas-api-tls\") pod \"maas-api-859d5d9c66-x7h8t\" (UID: \"69de6a4a-777e-4e70-9a9c-fe7198f9ca9c\") " pod="opendatahub/maas-api-859d5d9c66-x7h8t" Jun 08 18:39:26.619106 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:26.619106 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg8q6\" (UniqueName: \"kubernetes.io/projected/69de6a4a-777e-4e70-9a9c-fe7198f9ca9c-kube-api-access-wg8q6\") pod \"maas-api-859d5d9c66-x7h8t\" (UID: \"69de6a4a-777e-4e70-9a9c-fe7198f9ca9c\") " pod="opendatahub/maas-api-859d5d9c66-x7h8t" Jun 08 18:39:26.621716 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:26.621691 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69de6a4a-777e-4e70-9a9c-fe7198f9ca9c-maas-api-tls\") pod \"maas-api-859d5d9c66-x7h8t\" (UID: \"69de6a4a-777e-4e70-9a9c-fe7198f9ca9c\") " pod="opendatahub/maas-api-859d5d9c66-x7h8t" Jun 08 18:39:26.630850 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:26.630814 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg8q6\" (UniqueName: \"kubernetes.io/projected/69de6a4a-777e-4e70-9a9c-fe7198f9ca9c-kube-api-access-wg8q6\") pod \"maas-api-859d5d9c66-x7h8t\" (UID: \"69de6a4a-777e-4e70-9a9c-fe7198f9ca9c\") " pod="opendatahub/maas-api-859d5d9c66-x7h8t" Jun 08 18:39:26.719652 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:26.719567 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-859d5d9c66-x7h8t" Jun 08 18:39:26.874394 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:26.874244 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-859d5d9c66-x7h8t"] Jun 08 18:39:26.877385 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:39:26.877350 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69de6a4a_777e_4e70_9a9c_fe7198f9ca9c.slice/crio-af6510f1a831196c2f2c545c889e2be659f9daf16569cb82e8cdc7f629f3b56a WatchSource:0}: Error finding container af6510f1a831196c2f2c545c889e2be659f9daf16569cb82e8cdc7f629f3b56a: Status 404 returned error can't find the container with id af6510f1a831196c2f2c545c889e2be659f9daf16569cb82e8cdc7f629f3b56a Jun 08 18:39:26.878607 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:26.878589 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 18:39:27.150248 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:27.150211 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-gsxbr"] Jun 08 18:39:27.154881 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:27.154862 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-gsxbr" Jun 08 18:39:27.157489 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:27.157470 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 08 18:39:27.165663 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:27.165632 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-gsxbr"] Jun 08 18:39:27.215452 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:27.215413 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-mhrcq"] Jun 08 18:39:27.218838 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:27.218813 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-mhrcq" Jun 08 18:39:27.221282 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:27.221259 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-nq576\"" Jun 08 18:39:27.224566 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:27.224539 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwfks\" (UniqueName: \"kubernetes.io/projected/36ffb68e-458b-466e-9978-dfffb3611748-kube-api-access-vwfks\") pod \"payload-pre-processing-67d7689b65-gsxbr\" (UID: \"36ffb68e-458b-466e-9978-dfffb3611748\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-gsxbr" Jun 08 18:39:27.231135 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:27.231108 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-mhrcq"] Jun 08 18:39:27.326133 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:27.326095 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56zml\" (UniqueName: \"kubernetes.io/projected/863cd023-4ce4-4762-8918-4e4404b7b432-kube-api-access-56zml\") pod \"payload-processing-7c6c859784-mhrcq\" (UID: \"863cd023-4ce4-4762-8918-4e4404b7b432\") " pod="openshift-ingress/payload-processing-7c6c859784-mhrcq" Jun 08 18:39:27.326325 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:27.326161 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwfks\" (UniqueName: \"kubernetes.io/projected/36ffb68e-458b-466e-9978-dfffb3611748-kube-api-access-vwfks\") pod \"payload-pre-processing-67d7689b65-gsxbr\" (UID: \"36ffb68e-458b-466e-9978-dfffb3611748\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-gsxbr" Jun 08 18:39:27.340094 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:27.340060 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwfks\" (UniqueName: \"kubernetes.io/projected/36ffb68e-458b-466e-9978-dfffb3611748-kube-api-access-vwfks\") pod \"payload-pre-processing-67d7689b65-gsxbr\" (UID: \"36ffb68e-458b-466e-9978-dfffb3611748\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-gsxbr" Jun 08 18:39:27.427674 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:27.427588 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56zml\" (UniqueName: \"kubernetes.io/projected/863cd023-4ce4-4762-8918-4e4404b7b432-kube-api-access-56zml\") pod \"payload-processing-7c6c859784-mhrcq\" (UID: \"863cd023-4ce4-4762-8918-4e4404b7b432\") " pod="openshift-ingress/payload-processing-7c6c859784-mhrcq" Jun 08 18:39:27.441484 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:27.441450 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56zml\" (UniqueName: \"kubernetes.io/projected/863cd023-4ce4-4762-8918-4e4404b7b432-kube-api-access-56zml\") pod \"payload-processing-7c6c859784-mhrcq\" (UID: \"863cd023-4ce4-4762-8918-4e4404b7b432\") " pod="openshift-ingress/payload-processing-7c6c859784-mhrcq" Jun 08 18:39:27.467103 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:27.467071 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-gsxbr" Jun 08 18:39:27.530159 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:27.530088 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-mhrcq" Jun 08 18:39:27.584168 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:27.584082 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-859d5d9c66-x7h8t" event={"ID":"69de6a4a-777e-4e70-9a9c-fe7198f9ca9c","Type":"ContainerStarted","Data":"af6510f1a831196c2f2c545c889e2be659f9daf16569cb82e8cdc7f629f3b56a"} Jun 08 18:39:27.638559 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:27.638523 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-gsxbr"] Jun 08 18:39:27.645405 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:39:27.645171 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36ffb68e_458b_466e_9978_dfffb3611748.slice/crio-619b49661e01762ec95ed6b74cebf9df4b5d186a2ee2e066d86b0b0f842fb9a9 WatchSource:0}: Error finding container 619b49661e01762ec95ed6b74cebf9df4b5d186a2ee2e066d86b0b0f842fb9a9: Status 404 returned error can't find the container with id 619b49661e01762ec95ed6b74cebf9df4b5d186a2ee2e066d86b0b0f842fb9a9 Jun 08 18:39:27.702384 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:27.702295 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-mhrcq"] Jun 08 18:39:27.706571 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:39:27.706531 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod863cd023_4ce4_4762_8918_4e4404b7b432.slice/crio-40aadf7146bf9d0e0733488234b6430c167b514405ecfb2e8f7f63427e6c5e10 WatchSource:0}: Error finding container 40aadf7146bf9d0e0733488234b6430c167b514405ecfb2e8f7f63427e6c5e10: Status 404 returned error can't find the container with id 40aadf7146bf9d0e0733488234b6430c167b514405ecfb2e8f7f63427e6c5e10 Jun 08 18:39:28.594756 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:28.594723 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-mhrcq" event={"ID":"863cd023-4ce4-4762-8918-4e4404b7b432","Type":"ContainerStarted","Data":"40aadf7146bf9d0e0733488234b6430c167b514405ecfb2e8f7f63427e6c5e10"} Jun 08 18:39:28.596519 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:28.596477 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-gsxbr" event={"ID":"36ffb68e-458b-466e-9978-dfffb3611748","Type":"ContainerStarted","Data":"619b49661e01762ec95ed6b74cebf9df4b5d186a2ee2e066d86b0b0f842fb9a9"} Jun 08 18:39:29.602531 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:29.602427 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-859d5d9c66-x7h8t" event={"ID":"69de6a4a-777e-4e70-9a9c-fe7198f9ca9c","Type":"ContainerStarted","Data":"b416c9a8f11a46a855825fbbf41f6cb74492d07f1941a404e46c2eff23ac7ee1"} Jun 08 18:39:29.603001 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:29.602535 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-859d5d9c66-x7h8t" Jun 08 18:39:29.604050 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:29.604019 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-mhrcq" event={"ID":"863cd023-4ce4-4762-8918-4e4404b7b432","Type":"ContainerStarted","Data":"36cc9ece80b5afc72664f5d89c385dc6cbda0dd24b18cec780933db4d21b53ca"} Jun 08 18:39:29.604265 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:29.604136 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-mhrcq" Jun 08 18:39:29.605408 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:29.605386 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-gsxbr" event={"ID":"36ffb68e-458b-466e-9978-dfffb3611748","Type":"ContainerStarted","Data":"4a692498d557dd52a22dfe61edd1fd8934b551d1cd537744a9a9433ba2c10e82"} Jun 08 18:39:29.605515 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:29.605443 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-gsxbr" Jun 08 18:39:29.620638 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:29.620589 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-859d5d9c66-x7h8t" podStartSLOduration=1.959226666 podStartE2EDuration="3.620571664s" totalImagesPullingTime="1.661344998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:39:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:39:29.619614624 +0000 UTC m=+1603.474121422" watchObservedRunningTime="2026-06-08 18:39:29.620571664 +0000 UTC m=+1603.475078466" Jun 08 18:39:29.637998 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:29.637930 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7c6c859784-mhrcq" podStartSLOduration=1.185736225 podStartE2EDuration="2.637910733s" totalImagesPullingTime="1.452174508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:39:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:39:29.635443213 +0000 UTC m=+1603.489950458" watchObservedRunningTime="2026-06-08 18:39:29.637910733 +0000 UTC m=+1603.492417533" Jun 08 18:39:29.651802 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:29.651747 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-67d7689b65-gsxbr" podStartSLOduration=1.142387929 podStartE2EDuration="2.651732918s" totalImagesPullingTime="1.509344989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:39:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:39:29.649797826 +0000 UTC m=+1603.504304627" watchObservedRunningTime="2026-06-08 18:39:29.651732918 +0000 UTC m=+1603.506239723" Jun 08 18:39:35.615417 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:35.615379 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-859d5d9c66-x7h8t" Jun 08 18:39:40.611437 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:40.611408 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-mhrcq" Jun 08 18:39:40.611876 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:39:40.611725 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-gsxbr" Jun 08 18:40:38.976473 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:38.976433 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6"] Jun 08 18:40:38.980812 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:38.980788 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:40:38.982987 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:38.982947 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 08 18:40:38.983117 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:38.982950 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-flv6v\"" Jun 08 18:40:38.983300 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:38.983282 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 08 18:40:38.983612 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:38.983594 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 08 18:40:38.990106 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:38.990083 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6"] Jun 08 18:40:39.081109 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:39.081072 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fabe3edd-6407-41df-a632-8036a392f33a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6\" (UID: \"fabe3edd-6407-41df-a632-8036a392f33a\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:40:39.081109 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:39.081111 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fabe3edd-6407-41df-a632-8036a392f33a-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6\" (UID: \"fabe3edd-6407-41df-a632-8036a392f33a\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:40:39.081312 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:39.081135 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fabe3edd-6407-41df-a632-8036a392f33a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6\" (UID: \"fabe3edd-6407-41df-a632-8036a392f33a\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:40:39.081312 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:39.081178 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fabe3edd-6407-41df-a632-8036a392f33a-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6\" (UID: \"fabe3edd-6407-41df-a632-8036a392f33a\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:40:39.081312 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:39.081218 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fabe3edd-6407-41df-a632-8036a392f33a-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6\" (UID: \"fabe3edd-6407-41df-a632-8036a392f33a\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:40:39.081312 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:39.081288 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f4t8\" (UniqueName: \"kubernetes.io/projected/fabe3edd-6407-41df-a632-8036a392f33a-kube-api-access-4f4t8\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6\" (UID: \"fabe3edd-6407-41df-a632-8036a392f33a\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:40:39.182524 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:39.182488 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fabe3edd-6407-41df-a632-8036a392f33a-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6\" (UID: \"fabe3edd-6407-41df-a632-8036a392f33a\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:40:39.182688 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:39.182530 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fabe3edd-6407-41df-a632-8036a392f33a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6\" (UID: \"fabe3edd-6407-41df-a632-8036a392f33a\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:40:39.182688 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:39.182556 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fabe3edd-6407-41df-a632-8036a392f33a-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6\" (UID: \"fabe3edd-6407-41df-a632-8036a392f33a\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:40:39.182688 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:39.182613 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fabe3edd-6407-41df-a632-8036a392f33a-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6\" (UID: \"fabe3edd-6407-41df-a632-8036a392f33a\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:40:39.182833 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:39.182694 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f4t8\" (UniqueName: \"kubernetes.io/projected/fabe3edd-6407-41df-a632-8036a392f33a-kube-api-access-4f4t8\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6\" (UID: \"fabe3edd-6407-41df-a632-8036a392f33a\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:40:39.182833 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:39.182729 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fabe3edd-6407-41df-a632-8036a392f33a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6\" (UID: \"fabe3edd-6407-41df-a632-8036a392f33a\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:40:39.183103 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:39.183061 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fabe3edd-6407-41df-a632-8036a392f33a-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6\" (UID: \"fabe3edd-6407-41df-a632-8036a392f33a\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:40:39.183243 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:39.183108 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fabe3edd-6407-41df-a632-8036a392f33a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6\" (UID: \"fabe3edd-6407-41df-a632-8036a392f33a\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:40:39.183243 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:39.183080 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fabe3edd-6407-41df-a632-8036a392f33a-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6\" (UID: \"fabe3edd-6407-41df-a632-8036a392f33a\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:40:39.184922 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:39.184899 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fabe3edd-6407-41df-a632-8036a392f33a-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6\" (UID: \"fabe3edd-6407-41df-a632-8036a392f33a\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:40:39.185182 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:39.185165 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fabe3edd-6407-41df-a632-8036a392f33a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6\" (UID: \"fabe3edd-6407-41df-a632-8036a392f33a\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:40:39.190320 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:39.190300 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f4t8\" (UniqueName: \"kubernetes.io/projected/fabe3edd-6407-41df-a632-8036a392f33a-kube-api-access-4f4t8\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6\" (UID: \"fabe3edd-6407-41df-a632-8036a392f33a\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:40:39.292597 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:39.292498 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:40:39.433837 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:39.433803 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6"] Jun 08 18:40:39.437556 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:40:39.437524 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfabe3edd_6407_41df_a632_8036a392f33a.slice/crio-d96f571892969fe83521b448e4f3b2adafeb24c45069368a0dc1432ed0864841 WatchSource:0}: Error finding container d96f571892969fe83521b448e4f3b2adafeb24c45069368a0dc1432ed0864841: Status 404 returned error can't find the container with id d96f571892969fe83521b448e4f3b2adafeb24c45069368a0dc1432ed0864841 Jun 08 18:40:39.941010 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:39.940948 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" event={"ID":"fabe3edd-6407-41df-a632-8036a392f33a","Type":"ContainerStarted","Data":"d96f571892969fe83521b448e4f3b2adafeb24c45069368a0dc1432ed0864841"} Jun 08 18:40:44.973439 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:44.973404 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" event={"ID":"fabe3edd-6407-41df-a632-8036a392f33a","Type":"ContainerStarted","Data":"2bc00d9dc96f1b6a0e4acda3c577b69be19cb61555371e8f1ad6f667811c61eb"} Jun 08 18:40:59.263980 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.263875 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c"] Jun 08 18:40:59.269202 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.269177 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:40:59.271492 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.271465 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 08 18:40:59.278537 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.278514 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c"] Jun 08 18:40:59.366415 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.366378 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/701d9552-5c6e-4fd5-bfbd-5ab0083412f6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c\" (UID: \"701d9552-5c6e-4fd5-bfbd-5ab0083412f6\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:40:59.366415 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.366423 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/701d9552-5c6e-4fd5-bfbd-5ab0083412f6-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c\" (UID: \"701d9552-5c6e-4fd5-bfbd-5ab0083412f6\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:40:59.366635 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.366454 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/701d9552-5c6e-4fd5-bfbd-5ab0083412f6-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c\" (UID: \"701d9552-5c6e-4fd5-bfbd-5ab0083412f6\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:40:59.366635 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.366510 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klmfk\" (UniqueName: \"kubernetes.io/projected/701d9552-5c6e-4fd5-bfbd-5ab0083412f6-kube-api-access-klmfk\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c\" (UID: \"701d9552-5c6e-4fd5-bfbd-5ab0083412f6\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:40:59.366635 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.366572 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/701d9552-5c6e-4fd5-bfbd-5ab0083412f6-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c\" (UID: \"701d9552-5c6e-4fd5-bfbd-5ab0083412f6\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:40:59.366635 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.366596 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/701d9552-5c6e-4fd5-bfbd-5ab0083412f6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c\" (UID: \"701d9552-5c6e-4fd5-bfbd-5ab0083412f6\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:40:59.468002 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.467932 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/701d9552-5c6e-4fd5-bfbd-5ab0083412f6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c\" (UID: \"701d9552-5c6e-4fd5-bfbd-5ab0083412f6\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:40:59.468203 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.468018 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/701d9552-5c6e-4fd5-bfbd-5ab0083412f6-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c\" (UID: \"701d9552-5c6e-4fd5-bfbd-5ab0083412f6\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:40:59.468203 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.468054 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/701d9552-5c6e-4fd5-bfbd-5ab0083412f6-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c\" (UID: \"701d9552-5c6e-4fd5-bfbd-5ab0083412f6\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:40:59.468203 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.468089 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klmfk\" (UniqueName: \"kubernetes.io/projected/701d9552-5c6e-4fd5-bfbd-5ab0083412f6-kube-api-access-klmfk\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c\" (UID: \"701d9552-5c6e-4fd5-bfbd-5ab0083412f6\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:40:59.468203 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.468153 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/701d9552-5c6e-4fd5-bfbd-5ab0083412f6-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c\" (UID: \"701d9552-5c6e-4fd5-bfbd-5ab0083412f6\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:40:59.468396 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.468294 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/701d9552-5c6e-4fd5-bfbd-5ab0083412f6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c\" (UID: \"701d9552-5c6e-4fd5-bfbd-5ab0083412f6\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:40:59.468456 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.468425 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/701d9552-5c6e-4fd5-bfbd-5ab0083412f6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c\" (UID: \"701d9552-5c6e-4fd5-bfbd-5ab0083412f6\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:40:59.468456 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.468437 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/701d9552-5c6e-4fd5-bfbd-5ab0083412f6-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c\" (UID: \"701d9552-5c6e-4fd5-bfbd-5ab0083412f6\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:40:59.468568 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.468548 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/701d9552-5c6e-4fd5-bfbd-5ab0083412f6-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c\" (UID: \"701d9552-5c6e-4fd5-bfbd-5ab0083412f6\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:40:59.470295 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.470269 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/701d9552-5c6e-4fd5-bfbd-5ab0083412f6-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c\" (UID: \"701d9552-5c6e-4fd5-bfbd-5ab0083412f6\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:40:59.470706 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.470679 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/701d9552-5c6e-4fd5-bfbd-5ab0083412f6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c\" (UID: \"701d9552-5c6e-4fd5-bfbd-5ab0083412f6\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:40:59.475813 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.475790 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klmfk\" (UniqueName: \"kubernetes.io/projected/701d9552-5c6e-4fd5-bfbd-5ab0083412f6-kube-api-access-klmfk\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c\" (UID: \"701d9552-5c6e-4fd5-bfbd-5ab0083412f6\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:40:59.581334 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.581242 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:40:59.721397 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:40:59.721333 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c"] Jun 08 18:40:59.723796 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:40:59.723768 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod701d9552_5c6e_4fd5_bfbd_5ab0083412f6.slice/crio-640b38fd65785c3c9f2292aad383a7b00b67a0f8e1ad39e6e0c739c4c6100141 WatchSource:0}: Error finding container 640b38fd65785c3c9f2292aad383a7b00b67a0f8e1ad39e6e0c739c4c6100141: Status 404 returned error can't find the container with id 640b38fd65785c3c9f2292aad383a7b00b67a0f8e1ad39e6e0c739c4c6100141 Jun 08 18:41:00.050346 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:00.050311 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" event={"ID":"701d9552-5c6e-4fd5-bfbd-5ab0083412f6","Type":"ContainerStarted","Data":"846e16af5e6e63fdd4a534383381d9ffb4da6b5427c28e474a2348e526fa66a0"} Jun 08 18:41:00.050541 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:00.050364 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" event={"ID":"701d9552-5c6e-4fd5-bfbd-5ab0083412f6","Type":"ContainerStarted","Data":"640b38fd65785c3c9f2292aad383a7b00b67a0f8e1ad39e6e0c739c4c6100141"} Jun 08 18:41:02.062511 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:02.062474 2572 generic.go:356] "Generic (PLEG): container finished" podID="701d9552-5c6e-4fd5-bfbd-5ab0083412f6" containerID="846e16af5e6e63fdd4a534383381d9ffb4da6b5427c28e474a2348e526fa66a0" exitCode=0 Jun 08 18:41:02.062937 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:02.062545 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" event={"ID":"701d9552-5c6e-4fd5-bfbd-5ab0083412f6","Type":"ContainerDied","Data":"846e16af5e6e63fdd4a534383381d9ffb4da6b5427c28e474a2348e526fa66a0"} Jun 08 18:41:04.075690 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:04.075598 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" event={"ID":"701d9552-5c6e-4fd5-bfbd-5ab0083412f6","Type":"ContainerStarted","Data":"3b016ae47d37c1784c1a84bdabf09f1372186ef505c0c1717f3645cb6bdb8fdf"} Jun 08 18:41:04.094180 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:04.094120 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" podStartSLOduration=2.46341392 podStartE2EDuration="5.094098046s" totalImagesPullingTime="1.311605659s" totalInitContainerRuntime="1.319078467s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:41:04.092812433 +0000 UTC m=+1697.947319232" watchObservedRunningTime="2026-06-08 18:41:04.094098046 +0000 UTC m=+1697.948604846" Jun 08 18:41:09.582153 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:09.582106 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:41:09.582659 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:09.582186 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:41:09.594777 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:09.594743 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:41:10.122950 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:10.122922 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c" Jun 08 18:41:13.162278 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.162248 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l"] Jun 08 18:41:13.165840 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.165821 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:13.167875 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.167849 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 08 18:41:13.176653 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.176629 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l"] Jun 08 18:41:13.295734 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.295694 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l\" (UID: \"5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:13.295902 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.295743 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l\" (UID: \"5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:13.295902 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.295836 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l\" (UID: \"5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:13.295902 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.295881 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7htjv\" (UniqueName: \"kubernetes.io/projected/5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae-kube-api-access-7htjv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l\" (UID: \"5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:13.296092 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.295981 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l\" (UID: \"5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:13.296092 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.296007 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l\" (UID: \"5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:13.396733 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.396692 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l\" (UID: \"5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:13.396733 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.396735 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l\" (UID: \"5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:13.396989 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.396759 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l\" (UID: \"5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:13.396989 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.396792 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l\" (UID: \"5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:13.396989 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.396821 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l\" (UID: \"5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:13.396989 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.396859 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7htjv\" (UniqueName: \"kubernetes.io/projected/5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae-kube-api-access-7htjv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l\" (UID: \"5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:13.397258 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.397232 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l\" (UID: \"5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:13.397258 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.397250 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l\" (UID: \"5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:13.397406 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.397290 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l\" (UID: \"5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:13.399104 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.399081 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l\" (UID: \"5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:13.399405 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.399384 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l\" (UID: \"5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:13.403764 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.403739 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7htjv\" (UniqueName: \"kubernetes.io/projected/5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae-kube-api-access-7htjv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l\" (UID: \"5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:13.480714 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.480618 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:13.618027 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:13.617746 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l"] Jun 08 18:41:13.620203 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:41:13.620164 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ef68bfa_9cb3_4181_8ab9_fc828cfc29ae.slice/crio-79881f99aee23ec587df129a6ec344caa075b2d1e6dd3beffe99da5a9fc80e39 WatchSource:0}: Error finding container 79881f99aee23ec587df129a6ec344caa075b2d1e6dd3beffe99da5a9fc80e39: Status 404 returned error can't find the container with id 79881f99aee23ec587df129a6ec344caa075b2d1e6dd3beffe99da5a9fc80e39 Jun 08 18:41:14.134421 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:14.134379 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" event={"ID":"5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae","Type":"ContainerStarted","Data":"e233f55e97320c4409c808891ce49ff6c8c2081c17afef332492a7849ef57b44"} Jun 08 18:41:14.134590 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:14.134443 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" event={"ID":"5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae","Type":"ContainerStarted","Data":"79881f99aee23ec587df129a6ec344caa075b2d1e6dd3beffe99da5a9fc80e39"} Jun 08 18:41:15.140935 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:15.140889 2572 generic.go:356] "Generic (PLEG): container finished" podID="5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae" containerID="e233f55e97320c4409c808891ce49ff6c8c2081c17afef332492a7849ef57b44" exitCode=0 Jun 08 18:41:15.141340 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:15.140992 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" event={"ID":"5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae","Type":"ContainerDied","Data":"e233f55e97320c4409c808891ce49ff6c8c2081c17afef332492a7849ef57b44"} Jun 08 18:41:16.147953 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:16.147914 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" event={"ID":"5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae","Type":"ContainerStarted","Data":"bcf6dbfdf58471ad6783d267d3fadd00ac8bc5dc555c7fe96dc32135f5cb08a7"} Jun 08 18:41:16.166805 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:16.166746 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" podStartSLOduration=1.6127459210000001 podStartE2EDuration="3.166724735s" totalImagesPullingTime="216.229959ms" totalInitContainerRuntime="1.337748855s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:41:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:41:16.166336722 +0000 UTC m=+1710.020843521" watchObservedRunningTime="2026-06-08 18:41:16.166724735 +0000 UTC m=+1710.021231534" Jun 08 18:41:23.481039 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:23.480999 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:23.481458 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:23.481073 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:23.493577 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:23.493547 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:24.198419 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:24.198383 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l" Jun 08 18:41:39.064943 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.064906 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr"] Jun 08 18:41:39.076116 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.076088 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:41:39.078194 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.078168 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr"] Jun 08 18:41:39.078352 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.078180 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jun 08 18:41:39.140916 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.140880 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/47353b5b-e231-4ec1-8a3a-9ac8a87fc185-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kx8wr\" (UID: \"47353b5b-e231-4ec1-8a3a-9ac8a87fc185\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:41:39.141082 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.140936 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfqst\" (UniqueName: \"kubernetes.io/projected/47353b5b-e231-4ec1-8a3a-9ac8a87fc185-kube-api-access-pfqst\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kx8wr\" (UID: \"47353b5b-e231-4ec1-8a3a-9ac8a87fc185\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:41:39.141082 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.140980 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47353b5b-e231-4ec1-8a3a-9ac8a87fc185-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kx8wr\" (UID: \"47353b5b-e231-4ec1-8a3a-9ac8a87fc185\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:41:39.141082 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.141028 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47353b5b-e231-4ec1-8a3a-9ac8a87fc185-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kx8wr\" (UID: \"47353b5b-e231-4ec1-8a3a-9ac8a87fc185\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:41:39.141082 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.141043 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47353b5b-e231-4ec1-8a3a-9ac8a87fc185-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kx8wr\" (UID: \"47353b5b-e231-4ec1-8a3a-9ac8a87fc185\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:41:39.141223 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.141110 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47353b5b-e231-4ec1-8a3a-9ac8a87fc185-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kx8wr\" (UID: \"47353b5b-e231-4ec1-8a3a-9ac8a87fc185\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:41:39.241819 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.241780 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47353b5b-e231-4ec1-8a3a-9ac8a87fc185-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kx8wr\" (UID: \"47353b5b-e231-4ec1-8a3a-9ac8a87fc185\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:41:39.242028 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.241843 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/47353b5b-e231-4ec1-8a3a-9ac8a87fc185-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kx8wr\" (UID: \"47353b5b-e231-4ec1-8a3a-9ac8a87fc185\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:41:39.242028 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.241900 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfqst\" (UniqueName: \"kubernetes.io/projected/47353b5b-e231-4ec1-8a3a-9ac8a87fc185-kube-api-access-pfqst\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kx8wr\" (UID: \"47353b5b-e231-4ec1-8a3a-9ac8a87fc185\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:41:39.242028 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.241933 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47353b5b-e231-4ec1-8a3a-9ac8a87fc185-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kx8wr\" (UID: \"47353b5b-e231-4ec1-8a3a-9ac8a87fc185\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:41:39.242028 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.242016 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47353b5b-e231-4ec1-8a3a-9ac8a87fc185-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kx8wr\" (UID: \"47353b5b-e231-4ec1-8a3a-9ac8a87fc185\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:41:39.242246 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.242043 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47353b5b-e231-4ec1-8a3a-9ac8a87fc185-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kx8wr\" (UID: \"47353b5b-e231-4ec1-8a3a-9ac8a87fc185\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:41:39.242302 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.242271 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47353b5b-e231-4ec1-8a3a-9ac8a87fc185-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kx8wr\" (UID: \"47353b5b-e231-4ec1-8a3a-9ac8a87fc185\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:41:39.242366 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.242324 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/47353b5b-e231-4ec1-8a3a-9ac8a87fc185-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kx8wr\" (UID: \"47353b5b-e231-4ec1-8a3a-9ac8a87fc185\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:41:39.242366 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.242340 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47353b5b-e231-4ec1-8a3a-9ac8a87fc185-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kx8wr\" (UID: \"47353b5b-e231-4ec1-8a3a-9ac8a87fc185\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:41:39.244142 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.244118 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47353b5b-e231-4ec1-8a3a-9ac8a87fc185-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kx8wr\" (UID: \"47353b5b-e231-4ec1-8a3a-9ac8a87fc185\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:41:39.244488 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.244469 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47353b5b-e231-4ec1-8a3a-9ac8a87fc185-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kx8wr\" (UID: \"47353b5b-e231-4ec1-8a3a-9ac8a87fc185\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:41:39.249575 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.249557 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfqst\" (UniqueName: \"kubernetes.io/projected/47353b5b-e231-4ec1-8a3a-9ac8a87fc185-kube-api-access-pfqst\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-kx8wr\" (UID: \"47353b5b-e231-4ec1-8a3a-9ac8a87fc185\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:41:39.388215 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.388127 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:41:39.523764 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:39.523739 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr"] Jun 08 18:41:39.525974 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:41:39.525932 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47353b5b_e231_4ec1_8a3a_9ac8a87fc185.slice/crio-7aa22345f8ed0abf8074da61c6182cdb0dd23c18a470d0e2b1b26212f172747d WatchSource:0}: Error finding container 7aa22345f8ed0abf8074da61c6182cdb0dd23c18a470d0e2b1b26212f172747d: Status 404 returned error can't find the container with id 7aa22345f8ed0abf8074da61c6182cdb0dd23c18a470d0e2b1b26212f172747d Jun 08 18:41:40.267844 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:40.267804 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" event={"ID":"47353b5b-e231-4ec1-8a3a-9ac8a87fc185","Type":"ContainerStarted","Data":"4c683b8987cfea97eafcbeef31dae1d037acd2451595e400af0ee137e0cc276b"} Jun 08 18:41:40.268355 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:40.267860 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" event={"ID":"47353b5b-e231-4ec1-8a3a-9ac8a87fc185","Type":"ContainerStarted","Data":"7aa22345f8ed0abf8074da61c6182cdb0dd23c18a470d0e2b1b26212f172747d"} Jun 08 18:41:41.274459 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:41.274417 2572 generic.go:356] "Generic (PLEG): container finished" podID="47353b5b-e231-4ec1-8a3a-9ac8a87fc185" containerID="4c683b8987cfea97eafcbeef31dae1d037acd2451595e400af0ee137e0cc276b" exitCode=0 Jun 08 18:41:41.274867 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:41.274467 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" event={"ID":"47353b5b-e231-4ec1-8a3a-9ac8a87fc185","Type":"ContainerDied","Data":"4c683b8987cfea97eafcbeef31dae1d037acd2451595e400af0ee137e0cc276b"} Jun 08 18:41:42.282133 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:42.282093 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" event={"ID":"47353b5b-e231-4ec1-8a3a-9ac8a87fc185","Type":"ContainerStarted","Data":"ff428e42ba29de6d85c8c1e459df1fe47dde15720330dc43e5706e7ac730dc69"} Jun 08 18:41:42.304060 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:42.304003 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" podStartSLOduration=1.804154072 podStartE2EDuration="3.303984466s" totalImagesPullingTime="211.789107ms" totalInitContainerRuntime="1.288041287s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:41:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:41:42.301398765 +0000 UTC m=+1736.155905563" watchObservedRunningTime="2026-06-08 18:41:42.303984466 +0000 UTC m=+1736.158491264" Jun 08 18:41:46.303599 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:46.303513 2572 generic.go:356] "Generic (PLEG): container finished" podID="fabe3edd-6407-41df-a632-8036a392f33a" containerID="2bc00d9dc96f1b6a0e4acda3c577b69be19cb61555371e8f1ad6f667811c61eb" exitCode=0 Jun 08 18:41:46.304083 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:46.303593 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" event={"ID":"fabe3edd-6407-41df-a632-8036a392f33a","Type":"ContainerDied","Data":"2bc00d9dc96f1b6a0e4acda3c577b69be19cb61555371e8f1ad6f667811c61eb"} Jun 08 18:41:47.311592 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:47.311548 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" event={"ID":"fabe3edd-6407-41df-a632-8036a392f33a","Type":"ContainerStarted","Data":"1f14916cc4af5d38a4451d2b1ebc6e0513bd3e454a19ea781f5fefe5e5976ab8"} Jun 08 18:41:47.352109 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:47.352051 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" podStartSLOduration=2.640489558 podStartE2EDuration="1m9.352034076s" totalImagesPullingTime="4.827668631s" totalInitContainerRuntime="1m1.883875887s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:40:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:41:47.347882547 +0000 UTC m=+1741.202389344" watchObservedRunningTime="2026-06-08 18:41:47.352034076 +0000 UTC m=+1741.206540874" Jun 08 18:41:49.293036 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:49.292996 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:41:49.293450 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:49.293064 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:41:49.306169 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:49.306144 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:41:49.333794 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:49.333768 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6" Jun 08 18:41:49.389279 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:49.389243 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:41:49.389473 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:49.389303 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:41:49.402422 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:49.402397 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:41:50.339046 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:41:50.339020 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-kx8wr" Jun 08 18:42:46.665377 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:42:46.665349 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 18:42:46.667460 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:42:46.667435 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 18:44:40.671215 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:40.671178 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-548c5ccc8f-9ft5f"] Jun 08 18:44:40.671710 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:40.671364 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-548c5ccc8f-9ft5f" podUID="919ee8aa-7105-4658-a664-2ee87a9ebc76" containerName="manager" containerID="cri-o://643d60b57aaf1d3dfba0117eb135cf097430018fc4f1906956e1f4e6cd1a1597" gracePeriod=30 Jun 08 18:44:40.927279 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:40.927223 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-548c5ccc8f-9ft5f" Jun 08 18:44:41.020393 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:41.020365 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/919ee8aa-7105-4658-a664-2ee87a9ebc76-webhook-cert\") pod \"919ee8aa-7105-4658-a664-2ee87a9ebc76\" (UID: \"919ee8aa-7105-4658-a664-2ee87a9ebc76\") " Jun 08 18:44:41.020555 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:41.020444 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzkrc\" (UniqueName: \"kubernetes.io/projected/919ee8aa-7105-4658-a664-2ee87a9ebc76-kube-api-access-fzkrc\") pod \"919ee8aa-7105-4658-a664-2ee87a9ebc76\" (UID: \"919ee8aa-7105-4658-a664-2ee87a9ebc76\") " Jun 08 18:44:41.022483 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:41.022448 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/919ee8aa-7105-4658-a664-2ee87a9ebc76-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "919ee8aa-7105-4658-a664-2ee87a9ebc76" (UID: "919ee8aa-7105-4658-a664-2ee87a9ebc76"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 18:44:41.022593 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:41.022507 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/919ee8aa-7105-4658-a664-2ee87a9ebc76-kube-api-access-fzkrc" (OuterVolumeSpecName: "kube-api-access-fzkrc") pod "919ee8aa-7105-4658-a664-2ee87a9ebc76" (UID: "919ee8aa-7105-4658-a664-2ee87a9ebc76"). InnerVolumeSpecName "kube-api-access-fzkrc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:44:41.121758 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:41.121728 2572 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/919ee8aa-7105-4658-a664-2ee87a9ebc76-webhook-cert\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:44:41.121758 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:41.121758 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzkrc\" (UniqueName: \"kubernetes.io/projected/919ee8aa-7105-4658-a664-2ee87a9ebc76-kube-api-access-fzkrc\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:44:41.199456 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:41.199377 2572 generic.go:356] "Generic (PLEG): container finished" podID="919ee8aa-7105-4658-a664-2ee87a9ebc76" containerID="643d60b57aaf1d3dfba0117eb135cf097430018fc4f1906956e1f4e6cd1a1597" exitCode=0 Jun 08 18:44:41.199456 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:41.199412 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-548c5ccc8f-9ft5f" event={"ID":"919ee8aa-7105-4658-a664-2ee87a9ebc76","Type":"ContainerDied","Data":"643d60b57aaf1d3dfba0117eb135cf097430018fc4f1906956e1f4e6cd1a1597"} Jun 08 18:44:41.199456 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:41.199422 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-548c5ccc8f-9ft5f" Jun 08 18:44:41.199456 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:41.199452 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-548c5ccc8f-9ft5f" event={"ID":"919ee8aa-7105-4658-a664-2ee87a9ebc76","Type":"ContainerDied","Data":"13a96e13621c8d76f55e15219c8a5e90ad4c2a049d8461be9d917b5ca01c9116"} Jun 08 18:44:41.199722 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:41.199473 2572 scope.go:117] "RemoveContainer" containerID="643d60b57aaf1d3dfba0117eb135cf097430018fc4f1906956e1f4e6cd1a1597" Jun 08 18:44:41.208805 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:41.208786 2572 scope.go:117] "RemoveContainer" containerID="643d60b57aaf1d3dfba0117eb135cf097430018fc4f1906956e1f4e6cd1a1597" Jun 08 18:44:41.209082 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:44:41.209062 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"643d60b57aaf1d3dfba0117eb135cf097430018fc4f1906956e1f4e6cd1a1597\": container with ID starting with 643d60b57aaf1d3dfba0117eb135cf097430018fc4f1906956e1f4e6cd1a1597 not found: ID does not exist" containerID="643d60b57aaf1d3dfba0117eb135cf097430018fc4f1906956e1f4e6cd1a1597" Jun 08 18:44:41.209140 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:41.209091 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"643d60b57aaf1d3dfba0117eb135cf097430018fc4f1906956e1f4e6cd1a1597"} err="failed to get container status \"643d60b57aaf1d3dfba0117eb135cf097430018fc4f1906956e1f4e6cd1a1597\": rpc error: code = NotFound desc = could not find container \"643d60b57aaf1d3dfba0117eb135cf097430018fc4f1906956e1f4e6cd1a1597\": container with ID starting with 643d60b57aaf1d3dfba0117eb135cf097430018fc4f1906956e1f4e6cd1a1597 not found: ID does not exist" Jun 08 18:44:41.221087 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:41.221064 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-548c5ccc8f-9ft5f"] Jun 08 18:44:41.224911 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:41.224890 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-548c5ccc8f-9ft5f"] Jun 08 18:44:41.996075 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:41.996035 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-548c5ccc8f-bb8b8"] Jun 08 18:44:41.996592 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:41.996571 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="919ee8aa-7105-4658-a664-2ee87a9ebc76" containerName="manager" Jun 08 18:44:41.996680 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:41.996594 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="919ee8aa-7105-4658-a664-2ee87a9ebc76" containerName="manager" Jun 08 18:44:41.996758 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:41.996746 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="919ee8aa-7105-4658-a664-2ee87a9ebc76" containerName="manager" Jun 08 18:44:42.001221 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:42.001197 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-548c5ccc8f-bb8b8" Jun 08 18:44:42.003949 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:42.003925 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 08 18:44:42.004079 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:42.003984 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 08 18:44:42.004152 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:42.004094 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-sg4jj\"" Jun 08 18:44:42.006896 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:42.006875 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-548c5ccc8f-bb8b8"] Jun 08 18:44:42.128640 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:42.128609 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpmph\" (UniqueName: \"kubernetes.io/projected/4d8e0082-cf60-4972-997e-a48680449d8f-kube-api-access-tpmph\") pod \"maas-controller-548c5ccc8f-bb8b8\" (UID: \"4d8e0082-cf60-4972-997e-a48680449d8f\") " pod="opendatahub/maas-controller-548c5ccc8f-bb8b8" Jun 08 18:44:42.128820 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:42.128673 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4d8e0082-cf60-4972-997e-a48680449d8f-webhook-cert\") pod \"maas-controller-548c5ccc8f-bb8b8\" (UID: \"4d8e0082-cf60-4972-997e-a48680449d8f\") " pod="opendatahub/maas-controller-548c5ccc8f-bb8b8" Jun 08 18:44:42.230156 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:42.230124 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4d8e0082-cf60-4972-997e-a48680449d8f-webhook-cert\") pod \"maas-controller-548c5ccc8f-bb8b8\" (UID: \"4d8e0082-cf60-4972-997e-a48680449d8f\") " pod="opendatahub/maas-controller-548c5ccc8f-bb8b8" Jun 08 18:44:42.230357 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:42.230199 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpmph\" (UniqueName: \"kubernetes.io/projected/4d8e0082-cf60-4972-997e-a48680449d8f-kube-api-access-tpmph\") pod \"maas-controller-548c5ccc8f-bb8b8\" (UID: \"4d8e0082-cf60-4972-997e-a48680449d8f\") " pod="opendatahub/maas-controller-548c5ccc8f-bb8b8" Jun 08 18:44:42.232601 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:42.232575 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4d8e0082-cf60-4972-997e-a48680449d8f-webhook-cert\") pod \"maas-controller-548c5ccc8f-bb8b8\" (UID: \"4d8e0082-cf60-4972-997e-a48680449d8f\") " pod="opendatahub/maas-controller-548c5ccc8f-bb8b8" Jun 08 18:44:42.241428 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:42.241406 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpmph\" (UniqueName: \"kubernetes.io/projected/4d8e0082-cf60-4972-997e-a48680449d8f-kube-api-access-tpmph\") pod \"maas-controller-548c5ccc8f-bb8b8\" (UID: \"4d8e0082-cf60-4972-997e-a48680449d8f\") " pod="opendatahub/maas-controller-548c5ccc8f-bb8b8" Jun 08 18:44:42.313084 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:42.313011 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-548c5ccc8f-bb8b8" Jun 08 18:44:42.592313 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:42.592239 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="919ee8aa-7105-4658-a664-2ee87a9ebc76" path="/var/lib/kubelet/pods/919ee8aa-7105-4658-a664-2ee87a9ebc76/volumes" Jun 08 18:44:42.657708 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:42.657677 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-548c5ccc8f-bb8b8"] Jun 08 18:44:42.661263 ip-10-0-138-194 kubenswrapper[2572]: W0608 18:44:42.661234 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d8e0082_cf60_4972_997e_a48680449d8f.slice/crio-1a3e87f880b926da3f0c095c652599d7be566796f9daa90b72fcebe5df8a2b11 WatchSource:0}: Error finding container 1a3e87f880b926da3f0c095c652599d7be566796f9daa90b72fcebe5df8a2b11: Status 404 returned error can't find the container with id 1a3e87f880b926da3f0c095c652599d7be566796f9daa90b72fcebe5df8a2b11 Jun 08 18:44:42.662578 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:42.662557 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 18:44:43.211802 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:43.211768 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-548c5ccc8f-bb8b8" event={"ID":"4d8e0082-cf60-4972-997e-a48680449d8f","Type":"ContainerStarted","Data":"4364ed5473c5bd3d2ffdda5de508951aaf87e6949f7ca007307f2a9f039cf870"} Jun 08 18:44:43.212292 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:43.211852 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-548c5ccc8f-bb8b8" event={"ID":"4d8e0082-cf60-4972-997e-a48680449d8f","Type":"ContainerStarted","Data":"1a3e87f880b926da3f0c095c652599d7be566796f9daa90b72fcebe5df8a2b11"} Jun 08 18:44:43.212292 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:43.211951 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-548c5ccc8f-bb8b8" Jun 08 18:44:43.229531 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:43.229483 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-548c5ccc8f-bb8b8" podStartSLOduration=1.795013854 podStartE2EDuration="2.229467049s" totalImagesPullingTime="434.453195ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 18:44:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 18:44:43.227435042 +0000 UTC m=+1917.081941851" watchObservedRunningTime="2026-06-08 18:44:43.229467049 +0000 UTC m=+1917.083973846" Jun 08 18:44:54.223019 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:44:54.222989 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-548c5ccc8f-bb8b8" Jun 08 18:47:46.695066 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:47:46.695037 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 18:47:46.698525 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:47:46.698507 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 18:52:46.727589 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:52:46.727555 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 18:52:46.733773 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:52:46.733750 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 18:55:09.572458 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:55:09.572365 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7"] Jun 08 18:55:09.573192 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:55:09.572558 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7" podUID="c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2" containerName="manager" containerID="cri-o://673778bad28f3d9f2d416f212b352117cc6931a34a4c7526a6f00834ffce8dd5" gracePeriod=10 Jun 08 18:55:09.836048 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:55:09.835984 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7" Jun 08 18:55:09.950727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:55:09.950692 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgqjz\" (UniqueName: \"kubernetes.io/projected/c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2-kube-api-access-dgqjz\") pod \"c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2\" (UID: \"c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2\") " Jun 08 18:55:09.950727 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:55:09.950734 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2-extensions-socket-volume\") pod \"c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2\" (UID: \"c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2\") " Jun 08 18:55:09.951217 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:55:09.951195 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2" (UID: "c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 18:55:09.953048 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:55:09.953017 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2-kube-api-access-dgqjz" (OuterVolumeSpecName: "kube-api-access-dgqjz") pod "c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2" (UID: "c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2"). InnerVolumeSpecName "kube-api-access-dgqjz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 18:55:10.052343 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:55:10.052303 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgqjz\" (UniqueName: \"kubernetes.io/projected/c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2-kube-api-access-dgqjz\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:55:10.052343 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:55:10.052336 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2-extensions-socket-volume\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 18:55:10.382817 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:55:10.382785 2572 generic.go:356] "Generic (PLEG): container finished" podID="c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2" containerID="673778bad28f3d9f2d416f212b352117cc6931a34a4c7526a6f00834ffce8dd5" exitCode=0 Jun 08 18:55:10.383034 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:55:10.382851 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7" Jun 08 18:55:10.383034 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:55:10.382881 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7" event={"ID":"c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2","Type":"ContainerDied","Data":"673778bad28f3d9f2d416f212b352117cc6931a34a4c7526a6f00834ffce8dd5"} Jun 08 18:55:10.383034 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:55:10.382948 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7" event={"ID":"c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2","Type":"ContainerDied","Data":"3962eee0e9e43b16184ce5e02006a8ba4fc91b88150045a467e757dbed0b73da"} Jun 08 18:55:10.383034 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:55:10.382989 2572 scope.go:117] "RemoveContainer" containerID="673778bad28f3d9f2d416f212b352117cc6931a34a4c7526a6f00834ffce8dd5" Jun 08 18:55:10.392263 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:55:10.392246 2572 scope.go:117] "RemoveContainer" containerID="673778bad28f3d9f2d416f212b352117cc6931a34a4c7526a6f00834ffce8dd5" Jun 08 18:55:10.392493 ip-10-0-138-194 kubenswrapper[2572]: E0608 18:55:10.392476 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"673778bad28f3d9f2d416f212b352117cc6931a34a4c7526a6f00834ffce8dd5\": container with ID starting with 673778bad28f3d9f2d416f212b352117cc6931a34a4c7526a6f00834ffce8dd5 not found: ID does not exist" containerID="673778bad28f3d9f2d416f212b352117cc6931a34a4c7526a6f00834ffce8dd5" Jun 08 18:55:10.392539 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:55:10.392500 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"673778bad28f3d9f2d416f212b352117cc6931a34a4c7526a6f00834ffce8dd5"} err="failed to get container status \"673778bad28f3d9f2d416f212b352117cc6931a34a4c7526a6f00834ffce8dd5\": rpc error: code = NotFound desc = could not find container \"673778bad28f3d9f2d416f212b352117cc6931a34a4c7526a6f00834ffce8dd5\": container with ID starting with 673778bad28f3d9f2d416f212b352117cc6931a34a4c7526a6f00834ffce8dd5 not found: ID does not exist" Jun 08 18:55:10.408079 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:55:10.408054 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7"] Jun 08 18:55:10.411504 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:55:10.411478 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f9vz7"] Jun 08 18:55:10.592340 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:55:10.592299 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2" path="/var/lib/kubelet/pods/c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2/volumes" Jun 08 18:57:46.758370 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:57:46.758339 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 18:57:46.766071 ip-10-0-138-194 kubenswrapper[2572]: I0608 18:57:46.766048 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 19:02:46.789729 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:02:46.789703 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 19:02:46.797552 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:02:46.797531 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jpwxq_ac833fb5-5b86-4ab4-aa93-2441a8a30ff1/ovn-acl-logging/0.log" Jun 08 19:05:56.824239 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.824201 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p"] Jun 08 19:05:56.824848 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.824813 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2" containerName="manager" Jun 08 19:05:56.824848 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.824833 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2" containerName="manager" Jun 08 19:05:56.825038 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.824951 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8c4b40d-ca2f-4d1e-a66f-5d97e732b6c2" containerName="manager" Jun 08 19:05:56.828377 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.828353 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.830505 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.830480 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f5d60dca-openshift-default-dockercfg-w898j\"" Jun 08 19:05:56.839563 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.839536 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p"] Jun 08 19:05:56.857287 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.857255 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-workload-socket\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.857462 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.857307 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-data\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.857462 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.857332 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-credential-socket\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.857462 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.857372 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-workload-certs\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.857462 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.857444 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-envoy\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.857685 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.857487 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfxtx\" (UniqueName: \"kubernetes.io/projected/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-kube-api-access-pfxtx\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.857685 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.857549 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-token\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.857685 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.857595 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-podinfo\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.857685 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.857619 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istiod-ca-cert\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.958792 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.958748 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-workload-socket\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.959001 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.958807 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-data\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.959001 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.958833 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-credential-socket\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.959001 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.958869 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-workload-certs\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.959001 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.958903 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-envoy\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.959001 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.958932 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfxtx\" (UniqueName: \"kubernetes.io/projected/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-kube-api-access-pfxtx\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.959001 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.958997 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-token\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.959317 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.959025 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-podinfo\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.959317 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.959052 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istiod-ca-cert\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.959317 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.959251 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-workload-socket\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.959317 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.959274 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-data\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.959500 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.959420 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-credential-socket\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.959621 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.959593 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-workload-certs\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.959831 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.959814 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istiod-ca-cert\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.961413 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.961369 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-envoy\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.961659 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.961636 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-podinfo\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.966424 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.966405 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-token\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:56.966769 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:56.966744 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfxtx\" (UniqueName: \"kubernetes.io/projected/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-kube-api-access-pfxtx\") pod \"e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:57.141595 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:57.141508 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:05:57.290613 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:57.290580 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p"] Jun 08 19:05:57.292360 ip-10-0-138-194 kubenswrapper[2572]: W0608 19:05:57.292333 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb43c00e5_61d9_4eaa_b8d8_71cf911b03e7.slice/crio-48fcee8fb1576069d1d3587739c48993103209815546d2281d58d94a43e31e0e WatchSource:0}: Error finding container 48fcee8fb1576069d1d3587739c48993103209815546d2281d58d94a43e31e0e: Status 404 returned error can't find the container with id 48fcee8fb1576069d1d3587739c48993103209815546d2281d58d94a43e31e0e Jun 08 19:05:57.294849 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:57.294832 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 19:05:57.617219 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:05:57.617178 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" event={"ID":"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7","Type":"ContainerStarted","Data":"48fcee8fb1576069d1d3587739c48993103209815546d2281d58d94a43e31e0e"} Jun 08 19:06:01.269584 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:01.269542 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 08 19:06:01.269995 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:01.269632 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 08 19:06:01.269995 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:01.269681 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 08 19:06:01.639771 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:01.639675 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" event={"ID":"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7","Type":"ContainerStarted","Data":"58023ba081f56071566432e1eef9c5eb6b0a5d86985528c1c366cd8588a5a7f1"} Jun 08 19:06:01.663013 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:01.662928 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" podStartSLOduration=1.688643246 podStartE2EDuration="5.662906553s" totalImagesPullingTime="3.974263307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 19:05:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 19:06:01.66187494 +0000 UTC m=+3195.516381751" watchObservedRunningTime="2026-06-08 19:06:01.662906553 +0000 UTC m=+3195.517413352" Jun 08 19:06:02.142201 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:02.142167 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:06:02.147293 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:02.147268 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:06:02.645032 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:02.644998 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:06:02.646050 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:02.646030 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:06:04.346574 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:04.346541 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p"] Jun 08 19:06:04.655098 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:04.655062 2572 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" secret="" err="secret \"e2e-ait-f5d60dca-openshift-default-dockercfg-w898j\" not found" Jun 08 19:06:05.659431 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:05.659387 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" podUID="b43c00e5-61d9-4eaa-b8d8-71cf911b03e7" containerName="istio-proxy" containerID="cri-o://58023ba081f56071566432e1eef9c5eb6b0a5d86985528c1c366cd8588a5a7f1" gracePeriod=30 Jun 08 19:06:10.908669 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:10.908641 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:06:10.996626 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:10.996533 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istiod-ca-cert\") pod \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " Jun 08 19:06:10.996626 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:10.996585 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-data\") pod \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " Jun 08 19:06:10.996626 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:10.996611 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-podinfo\") pod \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " Jun 08 19:06:10.996882 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:10.996713 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-workload-socket\") pod \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " Jun 08 19:06:10.996882 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:10.996793 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-workload-certs\") pod \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " Jun 08 19:06:10.996882 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:10.996867 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-credential-socket\") pod \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " Jun 08 19:06:10.997079 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:10.996904 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-token\") pod \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " Jun 08 19:06:10.997079 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:10.996930 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-data" (OuterVolumeSpecName: "istio-data") pod "b43c00e5-61d9-4eaa-b8d8-71cf911b03e7" (UID: "b43c00e5-61d9-4eaa-b8d8-71cf911b03e7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 19:06:10.997079 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:10.996955 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b43c00e5-61d9-4eaa-b8d8-71cf911b03e7" (UID: "b43c00e5-61d9-4eaa-b8d8-71cf911b03e7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 19:06:10.997079 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:10.996988 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-envoy\") pod \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " Jun 08 19:06:10.997079 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:10.997038 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfxtx\" (UniqueName: \"kubernetes.io/projected/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-kube-api-access-pfxtx\") pod \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\" (UID: \"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7\") " Jun 08 19:06:10.997353 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:10.997220 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b43c00e5-61d9-4eaa-b8d8-71cf911b03e7" (UID: "b43c00e5-61d9-4eaa-b8d8-71cf911b03e7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 19:06:10.997353 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:10.997273 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b43c00e5-61d9-4eaa-b8d8-71cf911b03e7" (UID: "b43c00e5-61d9-4eaa-b8d8-71cf911b03e7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 19:06:10.997353 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:10.997314 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b43c00e5-61d9-4eaa-b8d8-71cf911b03e7" (UID: "b43c00e5-61d9-4eaa-b8d8-71cf911b03e7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 19:06:10.997488 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:10.997373 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-credential-socket\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 19:06:10.997488 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:10.997394 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istiod-ca-cert\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 19:06:10.997488 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:10.997434 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-data\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 19:06:10.999688 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:10.999651 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b43c00e5-61d9-4eaa-b8d8-71cf911b03e7" (UID: "b43c00e5-61d9-4eaa-b8d8-71cf911b03e7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 19:06:10.999816 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:10.999749 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-kube-api-access-pfxtx" (OuterVolumeSpecName: "kube-api-access-pfxtx") pod "b43c00e5-61d9-4eaa-b8d8-71cf911b03e7" (UID: "b43c00e5-61d9-4eaa-b8d8-71cf911b03e7"). InnerVolumeSpecName "kube-api-access-pfxtx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 19:06:10.999888 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:10.999850 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b43c00e5-61d9-4eaa-b8d8-71cf911b03e7" (UID: "b43c00e5-61d9-4eaa-b8d8-71cf911b03e7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jun 08 19:06:11.000013 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:10.999990 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-token" (OuterVolumeSpecName: "istio-token") pod "b43c00e5-61d9-4eaa-b8d8-71cf911b03e7" (UID: "b43c00e5-61d9-4eaa-b8d8-71cf911b03e7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 19:06:11.098171 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:11.098068 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-token\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 19:06:11.098171 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:11.098101 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-envoy\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 19:06:11.098171 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:11.098113 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfxtx\" (UniqueName: \"kubernetes.io/projected/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-kube-api-access-pfxtx\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 19:06:11.098171 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:11.098128 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-istio-podinfo\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 19:06:11.098171 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:11.098140 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-workload-socket\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 19:06:11.098171 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:11.098153 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7-workload-certs\") on node \"ip-10-0-138-194.ec2.internal\" DevicePath \"\"" Jun 08 19:06:11.691186 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:11.691152 2572 generic.go:356] "Generic (PLEG): container finished" podID="b43c00e5-61d9-4eaa-b8d8-71cf911b03e7" containerID="58023ba081f56071566432e1eef9c5eb6b0a5d86985528c1c366cd8588a5a7f1" exitCode=0 Jun 08 19:06:11.691350 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:11.691200 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" Jun 08 19:06:11.691350 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:11.691245 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" event={"ID":"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7","Type":"ContainerDied","Data":"58023ba081f56071566432e1eef9c5eb6b0a5d86985528c1c366cd8588a5a7f1"} Jun 08 19:06:11.691350 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:11.691300 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p" event={"ID":"b43c00e5-61d9-4eaa-b8d8-71cf911b03e7","Type":"ContainerDied","Data":"48fcee8fb1576069d1d3587739c48993103209815546d2281d58d94a43e31e0e"} Jun 08 19:06:11.691350 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:11.691328 2572 scope.go:117] "RemoveContainer" containerID="58023ba081f56071566432e1eef9c5eb6b0a5d86985528c1c366cd8588a5a7f1" Jun 08 19:06:11.700902 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:11.700886 2572 scope.go:117] "RemoveContainer" containerID="58023ba081f56071566432e1eef9c5eb6b0a5d86985528c1c366cd8588a5a7f1" Jun 08 19:06:11.701160 ip-10-0-138-194 kubenswrapper[2572]: E0608 19:06:11.701140 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58023ba081f56071566432e1eef9c5eb6b0a5d86985528c1c366cd8588a5a7f1\": container with ID starting with 58023ba081f56071566432e1eef9c5eb6b0a5d86985528c1c366cd8588a5a7f1 not found: ID does not exist" containerID="58023ba081f56071566432e1eef9c5eb6b0a5d86985528c1c366cd8588a5a7f1" Jun 08 19:06:11.701209 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:11.701168 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58023ba081f56071566432e1eef9c5eb6b0a5d86985528c1c366cd8588a5a7f1"} err="failed to get container status \"58023ba081f56071566432e1eef9c5eb6b0a5d86985528c1c366cd8588a5a7f1\": rpc error: code = NotFound desc = could not find container \"58023ba081f56071566432e1eef9c5eb6b0a5d86985528c1c366cd8588a5a7f1\": container with ID starting with 58023ba081f56071566432e1eef9c5eb6b0a5d86985528c1c366cd8588a5a7f1 not found: ID does not exist" Jun 08 19:06:11.711436 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:11.711411 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p"] Jun 08 19:06:11.715862 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:11.715842 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f5d60dca-openshift-default-8df986b7b-lp85p"] Jun 08 19:06:12.592728 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:12.592696 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b43c00e5-61d9-4eaa-b8d8-71cf911b03e7" path="/var/lib/kubelet/pods/b43c00e5-61d9-4eaa-b8d8-71cf911b03e7/volumes" Jun 08 19:06:21.331463 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:21.331428 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-7fbc7b66c7-vbxq5_30bdecef-135f-434f-ade4-126e684ab92d/nginx/0.log" Jun 08 19:06:21.456324 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:21.456295 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-pvcqw_53b4292e-3e7b-4c44-a681-ea0baac43d99/manager/0.log" Jun 08 19:06:21.598590 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:21.598512 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-z2vmb_c8da8b02-c7b3-4a1a-aa7c-7e35ca7fc0ce/manager/0.log" Jun 08 19:06:21.739130 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:21.739096 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-859d5d9c66-x7h8t_69de6a4a-777e-4e70-9a9c-fe7198f9ca9c/maas-api/0.log" Jun 08 19:06:22.112449 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:22.112411 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-548c5ccc8f-bb8b8_4d8e0082-cf60-4972-997e-a48680449d8f/manager/0.log" Jun 08 19:06:22.228939 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:22.228911 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-nplwv_52466ef3-77e8-45f8-ad25-f1f1b5612059/server/0.log" Jun 08 19:06:22.355408 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:22.355374 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-mqfxx_7b12b047-9e68-4273-a445-bfeeb6e06cbd/manager/0.log" Jun 08 19:06:22.753240 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:22.753211 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9564c99bf-xf4nh_32ac2387-9104-4c1a-a52f-74e10558d6a8/copy-manifests/0.log" Jun 08 19:06:22.760306 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:22.760271 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9564c99bf-xf4nh_32ac2387-9104-4c1a-a52f-74e10558d6a8/manager/0.log" Jun 08 19:06:22.873797 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:22.873754 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-7h5bg_2279cfbc-3808-4891-ad48-f5331edea3f3/postgres/0.log" Jun 08 19:06:24.448599 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:24.448560 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-hpdr9_479ea4f2-e2ef-4165-b4ec-5f489bab67cf/manager/0.log" Jun 08 19:06:25.556350 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:25.556317 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-x7j8h_8c1b3d87-99ba-4f72-b35c-ae1650daaa8c/discovery/0.log" Jun 08 19:06:25.673639 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:25.673605 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-ccd6bbfcc-5qb5d_36615221-c076-49fc-8c35-77476b334f68/kube-auth-proxy/0.log" Jun 08 19:06:26.030156 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:26.030120 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-gsxbr_36ffb68e-458b-466e-9978-dfffb3611748/payload-pre-processing/0.log" Jun 08 19:06:26.158218 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:26.158182 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-mhrcq_863cd023-4ce4-4762-8918-4e4404b7b432/payload-processing/0.log" Jun 08 19:06:26.286815 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:26.286736 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d8bbf5d9c-f27lp_685e569f-65a8-4c9b-b402-dc7ce33102ff/router/0.log" Jun 08 19:06:26.661691 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:26.661657 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6_fabe3edd-6407-41df-a632-8036a392f33a/storage-initializer/0.log" Jun 08 19:06:26.668973 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:26.668929 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-rk8l6_fabe3edd-6407-41df-a632-8036a392f33a/main/0.log" Jun 08 19:06:26.917646 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:26.917551 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c_701d9552-5c6e-4fd5-bfbd-5ab0083412f6/storage-initializer/0.log" Jun 08 19:06:26.925505 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:26.925480 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-7lc8c_701d9552-5c6e-4fd5-bfbd-5ab0083412f6/main/0.log" Jun 08 19:06:27.043856 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:27.043819 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l_5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae/main/0.log" Jun 08 19:06:27.051007 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:27.050981 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cp4p6l_5ef68bfa-9cb3-4181-8ab9-fc828cfc29ae/storage-initializer/0.log" Jun 08 19:06:27.297151 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:27.297066 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-kx8wr_47353b5b-e231-4ec1-8a3a-9ac8a87fc185/storage-initializer/0.log" Jun 08 19:06:27.305305 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:27.305275 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-kx8wr_47353b5b-e231-4ec1-8a3a-9ac8a87fc185/main/0.log" Jun 08 19:06:34.642497 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:34.642413 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-s8pt8_eca9e438-ae5a-4645-ab87-b8d44d7d35af/global-pull-secret-syncer/0.log" Jun 08 19:06:34.831805 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:34.831772 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-p892x_e27e2084-f792-4d85-9d63-99bb13520b73/konnectivity-agent/0.log" Jun 08 19:06:34.911038 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:34.911006 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-194.ec2.internal_d033c424e9151dca6b0d6a6e8144d8d3/haproxy/0.log" Jun 08 19:06:38.202360 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:38.202331 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-operators_servicemesh-operator3-57f65f65fb-bjm6s_af5833ca-6986-401a-a984-d253b6dbc2b2/sail-operator/0.log" Jun 08 19:06:39.497976 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:39.497881 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-hpdr9_479ea4f2-e2ef-4165-b4ec-5f489bab67cf/manager/0.log" Jun 08 19:06:42.076369 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:42.076334 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l8kvq_2df5c42a-be88-430d-9d73-ab199a589304/node-exporter/0.log" Jun 08 19:06:42.101139 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:42.101110 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l8kvq_2df5c42a-be88-430d-9d73-ab199a589304/kube-rbac-proxy/0.log" Jun 08 19:06:42.122953 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:42.122921 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l8kvq_2df5c42a-be88-430d-9d73-ab199a589304/init-textfile/0.log" Jun 08 19:06:42.252935 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:42.252906 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd75e56a-8538-4f11-8a9d-b011e66cc670/prometheus/0.log" Jun 08 19:06:42.274431 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:42.274404 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd75e56a-8538-4f11-8a9d-b011e66cc670/config-reloader/0.log" Jun 08 19:06:42.300887 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:42.300812 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd75e56a-8538-4f11-8a9d-b011e66cc670/thanos-sidecar/0.log" Jun 08 19:06:42.323866 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:42.323844 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd75e56a-8538-4f11-8a9d-b011e66cc670/kube-rbac-proxy-web/0.log" Jun 08 19:06:42.353823 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:42.353788 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd75e56a-8538-4f11-8a9d-b011e66cc670/kube-rbac-proxy/0.log" Jun 08 19:06:42.375811 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:42.375788 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd75e56a-8538-4f11-8a9d-b011e66cc670/kube-rbac-proxy-thanos/0.log" Jun 08 19:06:42.400494 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:42.400466 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd75e56a-8538-4f11-8a9d-b011e66cc670/init-config-reloader/0.log" Jun 08 19:06:46.621590 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:46.621553 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78dd696465-lnhdg_7782276f-b077-46b8-977e-2e7855e77764/volume-data-source-validator/0.log" Jun 08 19:06:46.979382 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:46.979348 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx"] Jun 08 19:06:46.979792 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:46.979778 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b43c00e5-61d9-4eaa-b8d8-71cf911b03e7" containerName="istio-proxy" Jun 08 19:06:46.979838 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:46.979793 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b43c00e5-61d9-4eaa-b8d8-71cf911b03e7" containerName="istio-proxy" Jun 08 19:06:46.979878 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:46.979871 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b43c00e5-61d9-4eaa-b8d8-71cf911b03e7" containerName="istio-proxy" Jun 08 19:06:46.984339 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:46.984316 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx" Jun 08 19:06:46.986338 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:46.986317 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pcs58\"/\"kube-root-ca.crt\"" Jun 08 19:06:46.986458 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:46.986443 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pcs58\"/\"openshift-service-ca.crt\"" Jun 08 19:06:46.986801 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:46.986781 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pcs58\"/\"default-dockercfg-qbhkp\"" Jun 08 19:06:46.990066 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:46.990046 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx"] Jun 08 19:06:47.125924 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:47.125893 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba34f5a8-37eb-46d4-ab69-8197c25d895d-lib-modules\") pod \"perf-node-gather-daemonset-9zpxx\" (UID: \"ba34f5a8-37eb-46d4-ab69-8197c25d895d\") " pod="openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx" Jun 08 19:06:47.126107 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:47.125982 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ba34f5a8-37eb-46d4-ab69-8197c25d895d-podres\") pod \"perf-node-gather-daemonset-9zpxx\" (UID: \"ba34f5a8-37eb-46d4-ab69-8197c25d895d\") " pod="openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx" Jun 08 19:06:47.126107 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:47.126007 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba34f5a8-37eb-46d4-ab69-8197c25d895d-sys\") pod \"perf-node-gather-daemonset-9zpxx\" (UID: \"ba34f5a8-37eb-46d4-ab69-8197c25d895d\") " pod="openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx" Jun 08 19:06:47.126107 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:47.126027 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb77v\" (UniqueName: \"kubernetes.io/projected/ba34f5a8-37eb-46d4-ab69-8197c25d895d-kube-api-access-kb77v\") pod \"perf-node-gather-daemonset-9zpxx\" (UID: \"ba34f5a8-37eb-46d4-ab69-8197c25d895d\") " pod="openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx" Jun 08 19:06:47.126107 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:47.126096 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ba34f5a8-37eb-46d4-ab69-8197c25d895d-proc\") pod \"perf-node-gather-daemonset-9zpxx\" (UID: \"ba34f5a8-37eb-46d4-ab69-8197c25d895d\") " pod="openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx" Jun 08 19:06:47.226731 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:47.226697 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ba34f5a8-37eb-46d4-ab69-8197c25d895d-podres\") pod \"perf-node-gather-daemonset-9zpxx\" (UID: \"ba34f5a8-37eb-46d4-ab69-8197c25d895d\") " pod="openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx" Jun 08 19:06:47.226731 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:47.226728 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba34f5a8-37eb-46d4-ab69-8197c25d895d-sys\") pod \"perf-node-gather-daemonset-9zpxx\" (UID: \"ba34f5a8-37eb-46d4-ab69-8197c25d895d\") " pod="openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx" Jun 08 19:06:47.226946 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:47.226750 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb77v\" (UniqueName: \"kubernetes.io/projected/ba34f5a8-37eb-46d4-ab69-8197c25d895d-kube-api-access-kb77v\") pod \"perf-node-gather-daemonset-9zpxx\" (UID: \"ba34f5a8-37eb-46d4-ab69-8197c25d895d\") " pod="openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx" Jun 08 19:06:47.226946 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:47.226801 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ba34f5a8-37eb-46d4-ab69-8197c25d895d-proc\") pod \"perf-node-gather-daemonset-9zpxx\" (UID: \"ba34f5a8-37eb-46d4-ab69-8197c25d895d\") " pod="openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx" Jun 08 19:06:47.226946 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:47.226831 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba34f5a8-37eb-46d4-ab69-8197c25d895d-sys\") pod \"perf-node-gather-daemonset-9zpxx\" (UID: \"ba34f5a8-37eb-46d4-ab69-8197c25d895d\") " pod="openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx" Jun 08 19:06:47.226946 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:47.226852 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ba34f5a8-37eb-46d4-ab69-8197c25d895d-podres\") pod \"perf-node-gather-daemonset-9zpxx\" (UID: \"ba34f5a8-37eb-46d4-ab69-8197c25d895d\") " pod="openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx" Jun 08 19:06:47.226946 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:47.226874 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba34f5a8-37eb-46d4-ab69-8197c25d895d-lib-modules\") pod \"perf-node-gather-daemonset-9zpxx\" (UID: \"ba34f5a8-37eb-46d4-ab69-8197c25d895d\") " pod="openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx" Jun 08 19:06:47.226946 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:47.226901 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ba34f5a8-37eb-46d4-ab69-8197c25d895d-proc\") pod \"perf-node-gather-daemonset-9zpxx\" (UID: \"ba34f5a8-37eb-46d4-ab69-8197c25d895d\") " pod="openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx" Jun 08 19:06:47.227176 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:47.227025 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba34f5a8-37eb-46d4-ab69-8197c25d895d-lib-modules\") pod \"perf-node-gather-daemonset-9zpxx\" (UID: \"ba34f5a8-37eb-46d4-ab69-8197c25d895d\") " pod="openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx" Jun 08 19:06:47.234086 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:47.234056 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb77v\" (UniqueName: \"kubernetes.io/projected/ba34f5a8-37eb-46d4-ab69-8197c25d895d-kube-api-access-kb77v\") pod \"perf-node-gather-daemonset-9zpxx\" (UID: \"ba34f5a8-37eb-46d4-ab69-8197c25d895d\") " pod="openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx" Jun 08 19:06:47.296091 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:47.296056 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx" Jun 08 19:06:47.434569 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:47.434544 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx"] Jun 08 19:06:47.436747 ip-10-0-138-194 kubenswrapper[2572]: W0608 19:06:47.436724 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podba34f5a8_37eb_46d4_ab69_8197c25d895d.slice/crio-2de1d00859c9a851a73c060b9b3f300eb2140d615fc42c3a90c1e49d9e0b39e4 WatchSource:0}: Error finding container 2de1d00859c9a851a73c060b9b3f300eb2140d615fc42c3a90c1e49d9e0b39e4: Status 404 returned error can't find the container with id 2de1d00859c9a851a73c060b9b3f300eb2140d615fc42c3a90c1e49d9e0b39e4 Jun 08 19:06:47.875213 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:47.875107 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx" event={"ID":"ba34f5a8-37eb-46d4-ab69-8197c25d895d","Type":"ContainerStarted","Data":"ad87d299574c9ba1a642ffe4d56c44caffbc43869d28c2e6e6d63b0963e7684b"} Jun 08 19:06:47.875213 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:47.875176 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx" event={"ID":"ba34f5a8-37eb-46d4-ab69-8197c25d895d","Type":"ContainerStarted","Data":"2de1d00859c9a851a73c060b9b3f300eb2140d615fc42c3a90c1e49d9e0b39e4"} Jun 08 19:06:47.875703 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:47.875220 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx" Jun 08 19:06:47.890807 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:47.890764 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx" podStartSLOduration=1.890749016 podStartE2EDuration="1.890749016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 19:06:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 19:06:47.888563784 +0000 UTC m=+3241.743070582" watchObservedRunningTime="2026-06-08 19:06:47.890749016 +0000 UTC m=+3241.745255864" Jun 08 19:06:48.060401 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:48.060370 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n4m82_c2665ffe-cccc-432a-85a7-a5244f0b2bc8/dns/0.log" Jun 08 19:06:48.083643 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:48.083612 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n4m82_c2665ffe-cccc-432a-85a7-a5244f0b2bc8/kube-rbac-proxy/0.log" Jun 08 19:06:48.160181 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:48.160148 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z77dq_1ad02886-8d99-4ad6-8103-a3eafea4af48/dns-node-resolver/0.log" Jun 08 19:06:48.710131 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:48.710096 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-dc69d89dc-lwb8p_ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b/registry/0.log" Jun 08 19:06:48.731431 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:48.731402 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-dc69d89dc-lwb8p_ebd76aeb-a1f1-4d59-b430-ac7d1eb6740b/registry/1.log" Jun 08 19:06:48.774635 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:48.774608 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bvg6f_445f7d76-73b2-4d8c-aad9-9195be1bd5fa/node-ca/0.log" Jun 08 19:06:49.892416 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:49.892382 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-x7j8h_8c1b3d87-99ba-4f72-b35c-ae1650daaa8c/discovery/0.log" Jun 08 19:06:49.910751 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:49.910729 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-ccd6bbfcc-5qb5d_36615221-c076-49fc-8c35-77476b334f68/kube-auth-proxy/0.log" Jun 08 19:06:50.061829 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:50.061790 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-gsxbr_36ffb68e-458b-466e-9978-dfffb3611748/payload-pre-processing/0.log" Jun 08 19:06:50.163181 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:50.163148 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-mhrcq_863cd023-4ce4-4762-8918-4e4404b7b432/payload-processing/0.log" Jun 08 19:06:50.183283 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:50.183256 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d8bbf5d9c-f27lp_685e569f-65a8-4c9b-b402-dc7ce33102ff/router/0.log" Jun 08 19:06:50.809912 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:50.809879 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bl784_0a6fe03e-3966-431d-99ff-43a27c4d0249/serve-healthcheck-canary/0.log" Jun 08 19:06:51.492531 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:51.492502 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gb9rc_ca9bcf18-14c9-4280-babf-4466a27fe082/kube-rbac-proxy/0.log" Jun 08 19:06:51.513329 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:51.513306 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gb9rc_ca9bcf18-14c9-4280-babf-4466a27fe082/exporter/0.log" Jun 08 19:06:51.534934 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:51.534909 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gb9rc_ca9bcf18-14c9-4280-babf-4466a27fe082/extractor/0.log" Jun 08 19:06:53.891805 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:53.891777 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pcs58/perf-node-gather-daemonset-9zpxx" Jun 08 19:06:53.959098 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:53.959062 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-7fbc7b66c7-vbxq5_30bdecef-135f-434f-ade4-126e684ab92d/nginx/0.log" Jun 08 19:06:53.983245 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:53.983212 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-pvcqw_53b4292e-3e7b-4c44-a681-ea0baac43d99/manager/0.log" Jun 08 19:06:54.118913 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:54.118877 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-z2vmb_c8da8b02-c7b3-4a1a-aa7c-7e35ca7fc0ce/manager/0.log" Jun 08 19:06:54.226172 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:54.226139 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-859d5d9c66-x7h8t_69de6a4a-777e-4e70-9a9c-fe7198f9ca9c/maas-api/0.log" Jun 08 19:06:54.393918 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:54.393880 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-548c5ccc8f-bb8b8_4d8e0082-cf60-4972-997e-a48680449d8f/manager/0.log" Jun 08 19:06:54.415719 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:54.415693 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-nplwv_52466ef3-77e8-45f8-ad25-f1f1b5612059/server/0.log" Jun 08 19:06:54.490666 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:54.490591 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-mqfxx_7b12b047-9e68-4273-a445-bfeeb6e06cbd/manager/0.log" Jun 08 19:06:54.718760 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:54.718736 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9564c99bf-xf4nh_32ac2387-9104-4c1a-a52f-74e10558d6a8/manager/0.log" Jun 08 19:06:54.742956 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:54.742887 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9564c99bf-xf4nh_32ac2387-9104-4c1a-a52f-74e10558d6a8/copy-manifests/0.log" Jun 08 19:06:54.766760 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:06:54.766733 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-7h5bg_2279cfbc-3808-4891-ad48-f5331edea3f3/postgres/0.log" Jun 08 19:07:00.950909 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:00.950879 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-x7j8h_8c1b3d87-99ba-4f72-b35c-ae1650daaa8c/discovery/0.log" Jun 08 19:07:00.961523 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:00.961492 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-ccd6bbfcc-5qb5d_36615221-c076-49fc-8c35-77476b334f68/kube-auth-proxy/0.log" Jun 08 19:07:01.085940 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:01.085905 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-gsxbr_36ffb68e-458b-466e-9978-dfffb3611748/payload-pre-processing/0.log" Jun 08 19:07:01.186763 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:01.186725 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-mhrcq_863cd023-4ce4-4762-8918-4e4404b7b432/payload-processing/0.log" Jun 08 19:07:01.200254 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:01.200231 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d8bbf5d9c-f27lp_685e569f-65a8-4c9b-b402-dc7ce33102ff/router/0.log" Jun 08 19:07:02.494766 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:02.494737 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-x7j8h_8c1b3d87-99ba-4f72-b35c-ae1650daaa8c/discovery/0.log" Jun 08 19:07:02.513297 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:02.513271 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-ccd6bbfcc-5qb5d_36615221-c076-49fc-8c35-77476b334f68/kube-auth-proxy/0.log" Jun 08 19:07:02.643352 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:02.643314 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-gsxbr_36ffb68e-458b-466e-9978-dfffb3611748/payload-pre-processing/0.log" Jun 08 19:07:02.735144 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:02.735106 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-mhrcq_863cd023-4ce4-4762-8918-4e4404b7b432/payload-processing/0.log" Jun 08 19:07:02.757374 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:02.757295 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d8bbf5d9c-f27lp_685e569f-65a8-4c9b-b402-dc7ce33102ff/router/0.log" Jun 08 19:07:04.212143 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:04.212114 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ztjlr_51fd7b9f-0366-434a-b567-33069764ed78/kube-multus-additional-cni-plugins/0.log" Jun 08 19:07:04.231108 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:04.231079 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ztjlr_51fd7b9f-0366-434a-b567-33069764ed78/egress-router-binary-copy/0.log" Jun 08 19:07:04.250983 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:04.250933 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ztjlr_51fd7b9f-0366-434a-b567-33069764ed78/cni-plugins/0.log" Jun 08 19:07:04.270611 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:04.270591 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ztjlr_51fd7b9f-0366-434a-b567-33069764ed78/bond-cni-plugin/0.log" Jun 08 19:07:04.290637 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:04.290614 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ztjlr_51fd7b9f-0366-434a-b567-33069764ed78/routeoverride-cni/0.log" Jun 08 19:07:04.314796 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:04.314771 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ztjlr_51fd7b9f-0366-434a-b567-33069764ed78/whereabouts-cni-bincopy/0.log" Jun 08 19:07:04.334568 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:04.334547 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ztjlr_51fd7b9f-0366-434a-b567-33069764ed78/whereabouts-cni/0.log" Jun 08 19:07:04.372673 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:04.372642 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qzwb8_9e2844f5-ca13-4cce-8fab-61322b119c50/kube-multus/0.log" Jun 08 19:07:04.474220 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:04.474106 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jjs4l_086390f2-8eb2-4903-a324-bf54451c435a/network-metrics-daemon/0.log" Jun 08 19:07:04.484694 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:04.484666 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-x7j8h_8c1b3d87-99ba-4f72-b35c-ae1650daaa8c/discovery/0.log" Jun 08 19:07:04.494594 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:04.494555 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jjs4l_086390f2-8eb2-4903-a324-bf54451c435a/kube-rbac-proxy/0.log" Jun 08 19:07:04.494789 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:04.494773 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-ccd6bbfcc-5qb5d_36615221-c076-49fc-8c35-77476b334f68/kube-auth-proxy/0.log" Jun 08 19:07:04.613813 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:04.613782 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-gsxbr_36ffb68e-458b-466e-9978-dfffb3611748/payload-pre-processing/0.log" Jun 08 19:07:04.703090 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:04.703059 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-mhrcq_863cd023-4ce4-4762-8918-4e4404b7b432/payload-processing/0.log" Jun 08 19:07:04.715594 ip-10-0-138-194 kubenswrapper[2572]: I0608 19:07:04.715572 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d8bbf5d9c-f27lp_685e569f-65a8-4c9b-b402-dc7ce33102ff/router/0.log"