Sep 08 02:21:44.346773 ip-10-0-141-33 systemd[1]: Starting Kubernetes Kubelet... Sep 08 02:21:44.782058 ip-10-0-141-33 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 02:21:44.782058 ip-10-0-141-33 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 08 02:21:44.782058 ip-10-0-141-33 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 02:21:44.782058 ip-10-0-141-33 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 08 02:21:44.782058 ip-10-0-141-33 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 02:21:44.785320 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.785220 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 08 02:21:44.792727 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792699 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 02:21:44.792727 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792719 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 02:21:44.792727 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792723 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 02:21:44.792727 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792726 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 02:21:44.792727 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792730 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 02:21:44.792727 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792733 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 02:21:44.792727 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792736 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 02:21:44.792727 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792739 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 02:21:44.793046 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792743 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 02:21:44.793046 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792746 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 02:21:44.793046 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792749 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 02:21:44.793046 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792752 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 02:21:44.793046 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792754 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 02:21:44.793046 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792757 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 02:21:44.793046 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792760 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 02:21:44.793046 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792762 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 02:21:44.793046 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792765 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 02:21:44.793046 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792768 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 02:21:44.793046 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792772 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 02:21:44.793046 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792774 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 02:21:44.793046 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792777 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 08 02:21:44.793046 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792779 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 02:21:44.793046 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792782 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 02:21:44.793046 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792784 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 02:21:44.793046 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792787 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 02:21:44.793046 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792789 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 02:21:44.793046 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792792 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 02:21:44.793046 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792795 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 02:21:44.793533 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792797 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 02:21:44.793533 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792800 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 02:21:44.793533 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792802 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 02:21:44.793533 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792805 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 02:21:44.793533 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792808 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 02:21:44.793533 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792813 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 02:21:44.793533 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792816 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 02:21:44.793533 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792819 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 02:21:44.793533 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792822 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 02:21:44.793533 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792826 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 02:21:44.793533 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792828 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 02:21:44.793533 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792831 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 02:21:44.793533 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792833 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 02:21:44.793533 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792836 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 02:21:44.793533 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792839 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 02:21:44.793533 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792841 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 02:21:44.793533 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792844 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 02:21:44.793533 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792847 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 02:21:44.793533 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792850 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 02:21:44.794020 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792852 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 02:21:44.794020 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792855 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 02:21:44.794020 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792858 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 02:21:44.794020 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792861 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 02:21:44.794020 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792863 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 02:21:44.794020 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792866 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 02:21:44.794020 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792869 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 02:21:44.794020 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792872 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 02:21:44.794020 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792875 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 02:21:44.794020 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792877 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 02:21:44.794020 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792880 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 02:21:44.794020 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792883 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 02:21:44.794020 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792885 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 02:21:44.794020 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792888 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 02:21:44.794020 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792890 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 02:21:44.794020 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792893 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 02:21:44.794020 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792895 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 02:21:44.794020 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792898 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 02:21:44.794020 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792900 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 02:21:44.794020 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792903 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 02:21:44.794515 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792905 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 02:21:44.794515 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792909 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 02:21:44.794515 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792912 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 02:21:44.794515 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792916 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 02:21:44.794515 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792919 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 02:21:44.794515 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792921 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 02:21:44.794515 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792925 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 02:21:44.794515 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792928 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 02:21:44.794515 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792932 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 02:21:44.794515 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792936 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 02:21:44.794515 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792939 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 02:21:44.794515 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792942 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 02:21:44.794515 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792945 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 02:21:44.794515 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792948 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 02:21:44.794515 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792952 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 02:21:44.794515 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792954 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 02:21:44.794515 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792958 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 02:21:44.794515 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792961 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 02:21:44.794515 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792974 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 02:21:44.794967 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792977 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 02:21:44.794967 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792980 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 02:21:44.794967 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792982 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 02:21:44.794967 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792985 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 02:21:44.794967 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792988 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 02:21:44.794967 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792990 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 02:21:44.794967 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792993 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 02:21:44.794967 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792995 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 02:21:44.794967 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.792998 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 02:21:44.794967 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793000 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 02:21:44.794967 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793003 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 02:21:44.794967 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793006 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 02:21:44.794967 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793010 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 02:21:44.794967 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793747 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 02:21:44.794967 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793754 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 02:21:44.794967 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793757 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 02:21:44.794967 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793761 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 02:21:44.794967 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793763 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 02:21:44.794967 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793766 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 02:21:44.794967 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793769 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 02:21:44.795432 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793771 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 02:21:44.795432 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793774 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 02:21:44.795432 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793776 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 02:21:44.795432 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793779 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 02:21:44.795432 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793782 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 02:21:44.795432 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793784 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 02:21:44.795432 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793787 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 02:21:44.795432 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793790 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 02:21:44.795432 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793792 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 02:21:44.795432 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793795 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 02:21:44.795432 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793798 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 02:21:44.795432 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793801 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 02:21:44.795432 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793803 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 02:21:44.795432 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793806 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 02:21:44.795432 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793808 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 02:21:44.795432 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793811 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 02:21:44.795432 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793814 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 02:21:44.795432 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793816 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 02:21:44.795432 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793819 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 02:21:44.795432 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793821 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 02:21:44.795948 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793824 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 02:21:44.795948 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793826 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 02:21:44.795948 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793829 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 02:21:44.795948 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793832 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 02:21:44.795948 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793836 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 02:21:44.795948 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793840 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 02:21:44.795948 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793843 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 02:21:44.795948 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793847 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 02:21:44.795948 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793850 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 02:21:44.795948 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793853 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 02:21:44.795948 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793855 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 02:21:44.795948 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793858 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 02:21:44.795948 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793861 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 02:21:44.795948 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793863 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 02:21:44.795948 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793866 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 02:21:44.795948 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793868 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 02:21:44.795948 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793871 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 02:21:44.795948 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793873 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 02:21:44.795948 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793876 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 02:21:44.796416 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793878 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 02:21:44.796416 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793881 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 08 02:21:44.796416 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793883 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 02:21:44.796416 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793886 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 02:21:44.796416 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793890 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 02:21:44.796416 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793892 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 02:21:44.796416 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793895 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 02:21:44.796416 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793897 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 02:21:44.796416 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793900 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 02:21:44.796416 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793903 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 02:21:44.796416 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793905 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 02:21:44.796416 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793908 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 02:21:44.796416 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793911 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 02:21:44.796416 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793913 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 02:21:44.796416 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793916 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 02:21:44.796416 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793919 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 02:21:44.796416 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793923 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 02:21:44.796416 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793925 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 02:21:44.796416 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793928 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 02:21:44.796416 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793931 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 02:21:44.796416 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793933 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 02:21:44.796942 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793936 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 02:21:44.796942 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793939 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 02:21:44.796942 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793941 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 02:21:44.796942 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793944 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 02:21:44.796942 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793947 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 02:21:44.796942 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793949 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 02:21:44.796942 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793952 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 02:21:44.796942 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793954 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 02:21:44.796942 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793957 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 02:21:44.796942 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793959 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 02:21:44.796942 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793961 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 02:21:44.796942 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793964 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 02:21:44.796942 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793966 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 02:21:44.796942 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793969 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 02:21:44.796942 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793971 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 02:21:44.796942 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793975 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 02:21:44.796942 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793979 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 02:21:44.796942 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793981 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 02:21:44.796942 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793984 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 02:21:44.796942 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793987 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 02:21:44.797440 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793989 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 02:21:44.797440 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793992 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 02:21:44.797440 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793994 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 02:21:44.797440 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.793997 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 02:21:44.797440 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794001 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 02:21:44.797440 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794004 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 02:21:44.797440 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794009 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 02:21:44.797440 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794013 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 02:21:44.797440 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794016 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 02:21:44.797440 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794019 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 02:21:44.797440 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794022 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 02:21:44.797440 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794025 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 02:21:44.797440 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794439 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 02:21:44.797440 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794444 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 02:21:44.797440 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794448 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 02:21:44.797440 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794452 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 02:21:44.797440 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794456 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 02:21:44.797440 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794475 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 02:21:44.797889 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794479 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 02:21:44.797889 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794482 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 02:21:44.797889 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794485 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 02:21:44.797889 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794488 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 02:21:44.797889 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794490 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 02:21:44.797889 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794493 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 02:21:44.797889 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794496 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 02:21:44.797889 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794498 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 02:21:44.797889 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794501 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 02:21:44.797889 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794503 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 02:21:44.797889 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794507 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 02:21:44.797889 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794510 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 02:21:44.797889 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794513 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 02:21:44.797889 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794515 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 02:21:44.797889 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794519 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 02:21:44.797889 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794523 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 02:21:44.797889 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794526 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 02:21:44.797889 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794529 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 02:21:44.797889 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794531 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 02:21:44.798373 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794534 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 02:21:44.798373 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794539 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 02:21:44.798373 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794542 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 02:21:44.798373 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794545 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 02:21:44.798373 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794547 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 02:21:44.798373 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794550 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 02:21:44.798373 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794552 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 02:21:44.798373 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794555 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 02:21:44.798373 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794557 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 02:21:44.798373 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794560 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 02:21:44.798373 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794562 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 02:21:44.798373 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794565 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 02:21:44.798373 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794567 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 02:21:44.798373 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794570 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 02:21:44.798373 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794572 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 02:21:44.798373 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794575 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 02:21:44.798373 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794577 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 02:21:44.798373 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794580 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 02:21:44.798373 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794582 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 02:21:44.798923 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794585 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 02:21:44.798923 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794588 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 02:21:44.798923 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794591 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 02:21:44.798923 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794593 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 02:21:44.798923 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794596 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 02:21:44.798923 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794599 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 02:21:44.798923 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794602 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 02:21:44.798923 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794605 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 02:21:44.798923 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794607 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 02:21:44.798923 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794609 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 02:21:44.798923 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794612 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 02:21:44.798923 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794617 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 02:21:44.798923 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794620 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 02:21:44.798923 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794623 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 02:21:44.798923 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794628 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 02:21:44.798923 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794630 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 02:21:44.798923 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794633 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 02:21:44.798923 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794636 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 02:21:44.798923 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794638 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 02:21:44.798923 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794641 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 02:21:44.799439 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794644 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 02:21:44.799439 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794646 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 02:21:44.799439 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794649 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 02:21:44.799439 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794651 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 02:21:44.799439 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794654 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 02:21:44.799439 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794657 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 02:21:44.799439 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794659 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 02:21:44.799439 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794662 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 02:21:44.799439 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794664 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 02:21:44.799439 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794667 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 02:21:44.799439 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794670 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 02:21:44.799439 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794672 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 02:21:44.799439 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794675 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 02:21:44.799439 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794678 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 02:21:44.799439 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794680 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 02:21:44.799439 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794683 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 02:21:44.799439 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794685 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 02:21:44.799439 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794689 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 02:21:44.799439 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794692 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 02:21:44.799439 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794694 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 02:21:44.799946 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794697 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 02:21:44.799946 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794700 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 02:21:44.799946 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794702 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 02:21:44.799946 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794705 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 02:21:44.799946 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794708 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 02:21:44.799946 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794710 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 02:21:44.799946 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794715 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 02:21:44.799946 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794718 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 02:21:44.799946 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794720 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 02:21:44.799946 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794723 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 02:21:44.799946 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794725 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 08 02:21:44.799946 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794728 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 02:21:44.799946 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794731 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 02:21:44.799946 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794733 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 02:21:44.799946 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.794736 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 02:21:44.799946 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794831 2560 flags.go:64] FLAG: --address="0.0.0.0" Sep 08 02:21:44.799946 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794871 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 08 02:21:44.799946 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794880 2560 flags.go:64] FLAG: --anonymous-auth="true" Sep 08 02:21:44.799946 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794886 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 08 02:21:44.799946 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794890 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 08 02:21:44.799946 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794893 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 08 02:21:44.799946 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794897 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794902 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794905 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794908 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794911 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794915 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794918 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794921 2560 flags.go:64] FLAG: --cgroup-root="" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794924 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794926 2560 flags.go:64] FLAG: --client-ca-file="" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794930 2560 flags.go:64] FLAG: --cloud-provider="external" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794933 2560 flags.go:64] FLAG: --cluster-dns="[]" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794938 2560 flags.go:64] FLAG: --cluster-domain="" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794941 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794944 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794947 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794950 2560 flags.go:64] FLAG: --container-log-max-files="5" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794957 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794960 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794963 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794966 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794970 2560 flags.go:64] FLAG: --contention-profiling="false" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794973 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794976 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 08 02:21:44.800499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794979 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794982 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794986 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794990 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794993 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794995 2560 flags.go:64] FLAG: --enable-load-reader="false" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.794998 2560 flags.go:64] FLAG: --enable-server="true" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795001 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795007 2560 flags.go:64] FLAG: --event-burst="100" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795010 2560 flags.go:64] FLAG: --event-qps="50" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795013 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795016 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795018 2560 flags.go:64] FLAG: --eviction-hard="" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795027 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795030 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795033 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795035 2560 flags.go:64] FLAG: --eviction-soft="" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795038 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795041 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795044 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795047 2560 flags.go:64] FLAG: --experimental-mounter-path="" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795050 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795053 2560 flags.go:64] FLAG: --fail-swap-on="true" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795056 2560 flags.go:64] FLAG: --feature-gates="" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795062 2560 flags.go:64] FLAG: --file-check-frequency="20s" Sep 08 02:21:44.801115 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795068 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795071 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795074 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795077 2560 flags.go:64] FLAG: --healthz-port="10248" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795080 2560 flags.go:64] FLAG: --help="false" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795083 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-141-33.ec2.internal" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795086 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795089 2560 flags.go:64] FLAG: --http-check-frequency="20s" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795092 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795096 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795099 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795102 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795105 2560 flags.go:64] FLAG: --image-service-endpoint="" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795107 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795110 2560 flags.go:64] FLAG: --kube-api-burst="100" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795113 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795116 2560 flags.go:64] FLAG: --kube-api-qps="50" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795119 2560 flags.go:64] FLAG: --kube-reserved="" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795122 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795125 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795128 2560 flags.go:64] FLAG: --kubelet-cgroups="" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795131 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795133 2560 flags.go:64] FLAG: --lock-file="" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795136 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 08 02:21:44.801762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795139 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 08 02:21:44.802343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795142 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 08 02:21:44.802343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795147 2560 flags.go:64] FLAG: --log-json-split-stream="false" Sep 08 02:21:44.802343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795150 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 08 02:21:44.802343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795153 2560 flags.go:64] FLAG: --log-text-split-stream="false" Sep 08 02:21:44.802343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795156 2560 flags.go:64] FLAG: --logging-format="text" Sep 08 02:21:44.802343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795159 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 08 02:21:44.802343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795162 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 08 02:21:44.802343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795168 2560 flags.go:64] FLAG: --manifest-url="" Sep 08 02:21:44.802343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795171 2560 flags.go:64] FLAG: --manifest-url-header="" Sep 08 02:21:44.802343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795175 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 08 02:21:44.802343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795178 2560 flags.go:64] FLAG: --max-open-files="1000000" Sep 08 02:21:44.802343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795182 2560 flags.go:64] FLAG: --max-pods="110" Sep 08 02:21:44.802343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795185 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 08 02:21:44.802343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795187 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 08 02:21:44.802343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795190 2560 flags.go:64] FLAG: --memory-manager-policy="None" Sep 08 02:21:44.802343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795193 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 08 02:21:44.802343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795197 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 08 02:21:44.802343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795199 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 08 02:21:44.802343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795202 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 08 02:21:44.802343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795210 2560 flags.go:64] FLAG: --node-status-max-images="50" Sep 08 02:21:44.802343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795213 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 08 02:21:44.802343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795216 2560 flags.go:64] FLAG: --oom-score-adj="-999" Sep 08 02:21:44.802343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795219 2560 flags.go:64] FLAG: --pod-cidr="" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795222 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795227 2560 flags.go:64] FLAG: --pod-manifest-path="" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795230 2560 flags.go:64] FLAG: --pod-max-pids="-1" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795233 2560 flags.go:64] FLAG: --pods-per-core="0" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795236 2560 flags.go:64] FLAG: --port="10250" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795239 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795242 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a47175bbaf1238e6" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795245 2560 flags.go:64] FLAG: --qos-reserved="" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795248 2560 flags.go:64] FLAG: --read-only-port="10255" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795250 2560 flags.go:64] FLAG: --register-node="true" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795253 2560 flags.go:64] FLAG: --register-with-taints="" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795257 2560 flags.go:64] FLAG: --registry-burst="10" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795261 2560 flags.go:64] FLAG: --registry-qps="5" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795264 2560 flags.go:64] FLAG: --reserved-cpus="" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795267 2560 flags.go:64] FLAG: --reserved-memory="" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795270 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795275 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795279 2560 flags.go:64] FLAG: --rotate-certificates="false" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795282 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795285 2560 flags.go:64] FLAG: --runonce="false" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795288 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795291 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795294 2560 flags.go:64] FLAG: --seccomp-default="false" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795297 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 08 02:21:44.802920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795299 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795302 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795306 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795309 2560 flags.go:64] FLAG: --storage-driver-password="root" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795312 2560 flags.go:64] FLAG: --storage-driver-secure="false" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795315 2560 flags.go:64] FLAG: --storage-driver-table="stats" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795318 2560 flags.go:64] FLAG: --storage-driver-user="root" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795321 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795324 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795326 2560 flags.go:64] FLAG: --system-cgroups="" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795329 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795334 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795337 2560 flags.go:64] FLAG: --tls-cert-file="" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795340 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795345 2560 flags.go:64] FLAG: --tls-min-version="" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795348 2560 flags.go:64] FLAG: --tls-private-key-file="" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795350 2560 flags.go:64] FLAG: --topology-manager-policy="none" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795353 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795356 2560 flags.go:64] FLAG: --topology-manager-scope="container" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795358 2560 flags.go:64] FLAG: --v="2" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795362 2560 flags.go:64] FLAG: --version="false" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795367 2560 flags.go:64] FLAG: --vmodule="" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795371 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.795374 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795505 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 02:21:44.803556 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795512 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 02:21:44.804155 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795516 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 02:21:44.804155 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795518 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 02:21:44.804155 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795521 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 02:21:44.804155 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795524 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 02:21:44.804155 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795526 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 02:21:44.804155 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795529 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 02:21:44.804155 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795532 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 02:21:44.804155 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795536 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 02:21:44.804155 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795538 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 02:21:44.804155 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795541 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 02:21:44.804155 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795544 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 02:21:44.804155 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795547 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 02:21:44.804155 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795549 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 02:21:44.804155 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795552 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 02:21:44.804155 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795555 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 02:21:44.804155 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795559 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 02:21:44.804155 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795562 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 02:21:44.804155 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795565 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 02:21:44.804155 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795568 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 02:21:44.804155 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795572 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 02:21:44.804686 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795574 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 02:21:44.804686 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795577 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 02:21:44.804686 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795580 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 02:21:44.804686 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795582 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 02:21:44.804686 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795586 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 02:21:44.804686 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795588 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 02:21:44.804686 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795591 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 02:21:44.804686 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795593 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 02:21:44.804686 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795596 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 02:21:44.804686 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795599 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 02:21:44.804686 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795603 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 02:21:44.804686 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795607 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 02:21:44.804686 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795610 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 02:21:44.804686 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795612 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 02:21:44.804686 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795615 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 02:21:44.804686 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795618 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 02:21:44.804686 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795621 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 02:21:44.804686 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795623 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 02:21:44.804686 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795626 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 02:21:44.804686 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795628 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 02:21:44.805183 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795631 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 02:21:44.805183 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795633 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 02:21:44.805183 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795636 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 02:21:44.805183 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795638 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 02:21:44.805183 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795641 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 02:21:44.805183 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795644 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 02:21:44.805183 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795646 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 02:21:44.805183 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795649 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 02:21:44.805183 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795651 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 02:21:44.805183 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795654 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 02:21:44.805183 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795657 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 02:21:44.805183 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795659 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 02:21:44.805183 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795662 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 02:21:44.805183 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795664 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 02:21:44.805183 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795667 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 02:21:44.805183 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795669 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 02:21:44.805183 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795672 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 02:21:44.805183 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795674 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 02:21:44.805183 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795677 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 02:21:44.805183 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795679 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 02:21:44.805687 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795682 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 02:21:44.805687 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795685 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 02:21:44.805687 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795689 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 02:21:44.805687 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795693 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 02:21:44.805687 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795696 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 02:21:44.805687 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795699 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 02:21:44.805687 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795701 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 02:21:44.805687 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795704 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 02:21:44.805687 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795707 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 02:21:44.805687 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795709 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 02:21:44.805687 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795712 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 02:21:44.805687 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795715 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 02:21:44.805687 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795717 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 02:21:44.805687 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795720 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 02:21:44.805687 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795722 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 02:21:44.805687 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795725 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 02:21:44.805687 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795727 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 02:21:44.805687 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795730 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 02:21:44.805687 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795733 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 02:21:44.805687 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795736 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 02:21:44.806174 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795738 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 02:21:44.806174 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795742 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 02:21:44.806174 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795746 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 02:21:44.806174 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795749 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 02:21:44.806174 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795752 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 02:21:44.806174 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795755 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 02:21:44.806174 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795757 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 02:21:44.806174 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795760 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 02:21:44.806174 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795763 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 02:21:44.806174 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795765 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 02:21:44.806174 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795768 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 08 02:21:44.806174 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795771 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 02:21:44.806174 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795775 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 02:21:44.806174 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795778 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 02:21:44.806174 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795783 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 02:21:44.806174 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795787 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 02:21:44.806174 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.795790 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 02:21:44.806615 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.796264 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 02:21:44.806615 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.803063 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 08 02:21:44.806615 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.803082 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 08 02:21:44.806615 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.803825 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 08 02:21:44.806615 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.803833 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 08 02:21:44.806615 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803907 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 02:21:44.806615 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803913 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 02:21:44.806615 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803916 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 02:21:44.806615 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803918 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 02:21:44.806615 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803922 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 02:21:44.806615 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803925 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 02:21:44.806615 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803927 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 02:21:44.806615 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803930 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 02:21:44.806615 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803932 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 02:21:44.806615 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803935 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 02:21:44.806989 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803938 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 02:21:44.806989 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803940 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 02:21:44.806989 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803943 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 02:21:44.806989 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803945 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 02:21:44.806989 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803948 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 02:21:44.806989 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803951 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 02:21:44.806989 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803955 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 02:21:44.806989 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803958 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 02:21:44.806989 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803961 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 02:21:44.806989 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803964 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 02:21:44.806989 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803967 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 02:21:44.806989 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803970 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 02:21:44.806989 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803972 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 02:21:44.806989 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803975 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 02:21:44.806989 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803978 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 02:21:44.806989 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803980 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 02:21:44.806989 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803982 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 02:21:44.806989 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803985 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 02:21:44.806989 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803987 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 02:21:44.806989 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803991 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 02:21:44.807480 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803995 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 02:21:44.807480 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.803998 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 02:21:44.807480 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804000 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 02:21:44.807480 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804002 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 02:21:44.807480 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804005 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 02:21:44.807480 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804009 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 02:21:44.807480 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804012 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 02:21:44.807480 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804015 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 02:21:44.807480 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804018 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 02:21:44.807480 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804021 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 02:21:44.807480 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804023 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 02:21:44.807480 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804026 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 02:21:44.807480 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804028 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 02:21:44.807480 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804031 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 02:21:44.807480 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804034 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 02:21:44.807480 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804036 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 02:21:44.807480 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804039 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 02:21:44.807480 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804042 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 02:21:44.807480 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804044 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 02:21:44.807480 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804047 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 02:21:44.808011 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804049 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 02:21:44.808011 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804052 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 02:21:44.808011 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804055 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 02:21:44.808011 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804057 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 02:21:44.808011 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804059 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 02:21:44.808011 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804062 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 02:21:44.808011 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804065 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 02:21:44.808011 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804067 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 02:21:44.808011 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804070 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 02:21:44.808011 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804072 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 02:21:44.808011 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804075 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 02:21:44.808011 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804078 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 02:21:44.808011 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804081 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 02:21:44.808011 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804084 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 02:21:44.808011 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804087 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 02:21:44.808011 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804089 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 02:21:44.808011 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804092 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 02:21:44.808011 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804095 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 02:21:44.808011 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804097 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 02:21:44.808011 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804100 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 02:21:44.808619 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804102 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 02:21:44.808619 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804105 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 02:21:44.808619 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804107 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 02:21:44.808619 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804110 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 02:21:44.808619 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804112 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 02:21:44.808619 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804115 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 02:21:44.808619 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804117 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 02:21:44.808619 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804120 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 02:21:44.808619 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804122 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 02:21:44.808619 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804125 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 02:21:44.808619 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804127 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 02:21:44.808619 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804130 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 02:21:44.808619 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804132 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 02:21:44.808619 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804135 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 02:21:44.808619 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804137 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 08 02:21:44.808619 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804140 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 02:21:44.808619 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804142 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 02:21:44.808619 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804145 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 02:21:44.808619 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804147 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 02:21:44.809087 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804151 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 02:21:44.809087 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804154 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 02:21:44.809087 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804156 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 02:21:44.809087 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804159 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 02:21:44.809087 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804162 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 02:21:44.809087 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804165 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 02:21:44.809087 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804168 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 02:21:44.809087 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804171 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 02:21:44.809087 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804173 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 02:21:44.809087 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804176 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 02:21:44.809087 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.804181 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 02:21:44.809087 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804305 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 02:21:44.809087 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804309 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 02:21:44.809412 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804314 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 02:21:44.809412 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804318 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 02:21:44.809412 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804321 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 02:21:44.809412 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804324 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 02:21:44.809412 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804326 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 02:21:44.809412 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804329 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 02:21:44.809412 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804332 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 02:21:44.809412 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804335 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 02:21:44.809412 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804337 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 02:21:44.809412 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804340 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 02:21:44.809412 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804342 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 02:21:44.809412 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804345 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 02:21:44.809412 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804347 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 02:21:44.809412 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804350 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 02:21:44.809412 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804352 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 02:21:44.809412 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804356 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 02:21:44.809412 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804358 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 02:21:44.809412 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804361 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 02:21:44.809412 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804363 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 02:21:44.809412 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804366 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 02:21:44.809919 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804369 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 02:21:44.809919 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804371 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 02:21:44.809919 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804374 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 02:21:44.809919 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804376 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 02:21:44.809919 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804379 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 02:21:44.809919 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804381 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 02:21:44.809919 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804384 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 02:21:44.809919 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804387 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 02:21:44.809919 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804389 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 02:21:44.809919 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804392 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 02:21:44.809919 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804394 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 02:21:44.809919 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804397 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 02:21:44.809919 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804399 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 02:21:44.809919 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804402 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 02:21:44.809919 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804405 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 02:21:44.809919 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804407 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 02:21:44.809919 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804410 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 02:21:44.809919 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804412 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 02:21:44.809919 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804414 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 02:21:44.809919 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804417 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 02:21:44.810391 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804419 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 02:21:44.810391 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804422 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 02:21:44.810391 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804425 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 02:21:44.810391 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804427 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 02:21:44.810391 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804430 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 02:21:44.810391 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804432 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 02:21:44.810391 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804434 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 02:21:44.810391 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804438 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 02:21:44.810391 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804441 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 02:21:44.810391 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804443 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 02:21:44.810391 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804446 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 02:21:44.810391 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804448 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 02:21:44.810391 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804451 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 02:21:44.810391 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804453 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 02:21:44.810391 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804456 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 02:21:44.810391 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804470 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 02:21:44.810391 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804473 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 02:21:44.810391 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804476 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 02:21:44.810391 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804478 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 02:21:44.810391 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804481 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 02:21:44.810892 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804484 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 02:21:44.810892 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804487 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 02:21:44.810892 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804489 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 02:21:44.810892 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804492 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 02:21:44.810892 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804495 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 02:21:44.810892 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804497 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 02:21:44.810892 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804501 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 02:21:44.810892 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804504 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 02:21:44.810892 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804507 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 02:21:44.810892 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804510 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 02:21:44.810892 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804512 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 02:21:44.810892 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804515 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 02:21:44.810892 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804518 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 02:21:44.810892 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804521 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 02:21:44.810892 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804524 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 02:21:44.810892 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804526 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 02:21:44.810892 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804529 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 02:21:44.810892 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804532 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 02:21:44.810892 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804535 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 02:21:44.811354 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804538 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 02:21:44.811354 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804540 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 02:21:44.811354 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804543 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 02:21:44.811354 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804546 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 02:21:44.811354 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804548 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 02:21:44.811354 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804551 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 02:21:44.811354 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804553 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 02:21:44.811354 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804556 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 02:21:44.811354 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804558 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 02:21:44.811354 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804561 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 08 02:21:44.811354 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804563 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 02:21:44.811354 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804566 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 02:21:44.811354 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804568 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 02:21:44.811354 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804572 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 02:21:44.811354 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804575 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 02:21:44.811354 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804579 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 02:21:44.811354 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804582 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 02:21:44.811354 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:44.804584 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 02:21:44.811870 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.804588 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 02:21:44.811870 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.804708 2560 server.go:962] "Client rotation is on, will bootstrap in background" Sep 08 02:21:44.811870 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.808818 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 08 02:21:44.811870 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.809656 2560 server.go:1019] "Starting client certificate rotation" Sep 08 02:21:44.811870 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.809762 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 08 02:21:44.811870 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.809804 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 08 02:21:44.813415 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.813400 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 08 02:21:44.814658 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.814641 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 08 02:21:44.828021 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.828001 2560 log.go:25] "Validated CRI v1 runtime API" Sep 08 02:21:44.831234 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.831217 2560 log.go:25] "Validated CRI v1 image API" Sep 08 02:21:44.832155 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.832133 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 08 02:21:44.837092 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.837073 2560 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 96c2c06d-70a4-4ffd-9cc3-aad7813a3ce2:/dev/nvme0n1p3 f3a522ad-13e8-4547-8ee8-1c971cbc74e5:/dev/nvme0n1p4] Sep 08 02:21:44.837174 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.837091 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 08 02:21:44.839636 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.839614 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 08 02:21:44.841998 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.841880 2560 manager.go:216] Machine: {Timestamp:2026-09-08 02:21:44.840708524 +0000 UTC m=+0.372353379 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101613 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2727ef8f5f91cc4fd6df737ac20487 SystemUUID:ec2727ef-8f5f-91cc-4fd6-df737ac20487 BootID:4d44f0cc-d6e5-4706-bf46-59ba5f6a9cd7 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:50:19:da:01:d3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:50:19:da:01:d3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:aa:5e:b1:ef:02:fc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 08 02:21:44.841998 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.841993 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 08 02:21:44.842114 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.842077 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 08 02:21:44.845406 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.845375 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 08 02:21:44.845562 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.845409 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-33.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 08 02:21:44.846316 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.846305 2560 topology_manager.go:138] "Creating topology manager with none policy" Sep 08 02:21:44.846352 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.846319 2560 container_manager_linux.go:309] "Creating device plugin manager" Sep 08 02:21:44.846352 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.846327 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 08 02:21:44.847132 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.847121 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 08 02:21:44.848285 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.848275 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 08 02:21:44.851015 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.850999 2560 state_mem.go:36] "Initialized new in-memory state store" Sep 08 02:21:44.851141 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.851130 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 08 02:21:44.851174 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.851163 2560 kubelet.go:486] "Attempting to sync node with API server" Sep 08 02:21:44.851211 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.851177 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 08 02:21:44.851211 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.851200 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 08 02:21:44.851268 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.851214 2560 kubelet.go:398] "Adding apiserver pod source" Sep 08 02:21:44.851268 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.851226 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 08 02:21:44.851741 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.851727 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 08 02:21:44.851798 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.851753 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 08 02:21:44.858039 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.858018 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 08 02:21:44.859944 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.859931 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 08 02:21:44.860000 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.859960 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 08 02:21:44.860134 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.860124 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 08 02:21:44.860167 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.860138 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 08 02:21:44.860167 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.860144 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 08 02:21:44.860167 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.860149 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 08 02:21:44.860167 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.860168 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 08 02:21:44.860273 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.860174 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 08 02:21:44.860608 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.860598 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 08 02:21:44.860608 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.860608 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 08 02:21:44.860668 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.860616 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 08 02:21:44.860668 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.860621 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 08 02:21:44.860668 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.860635 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 08 02:21:44.860668 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.860642 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 08 02:21:44.861881 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:44.861861 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-33.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 08 02:21:44.861920 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:44.861851 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 08 02:21:44.862343 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.862330 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 08 02:21:44.862377 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.862345 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 08 02:21:44.865921 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.865908 2560 server.go:1268] "Started kubelet" Sep 08 02:21:44.866068 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.865997 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 08 02:21:44.872282 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.872229 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 08 02:21:44.872374 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.872313 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 08 02:21:44.873217 ip-10-0-141-33 systemd[1]: Started Kubernetes Kubelet. Sep 08 02:21:44.873478 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.873368 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6btfk" Sep 08 02:21:44.873753 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.873733 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 08 02:21:44.874661 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.874641 2560 server.go:310] "Adding debug handlers to kubelet server" Sep 08 02:21:44.878776 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.878757 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6btfk" Sep 08 02:21:44.879371 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:44.879345 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 08 02:21:44.879797 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.879776 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 08 02:21:44.880146 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.880127 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-33.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 08 02:21:44.880504 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.880278 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 08 02:21:44.880564 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:44.880534 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Sep 08 02:21:44.880961 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.880942 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 08 02:21:44.881176 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.881162 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 08 02:21:44.881260 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.881179 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 08 02:21:44.881312 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.881271 2560 reconstruct.go:97] "Volume reconstruction finished" Sep 08 02:21:44.881312 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.881279 2560 reconciler.go:29] "Reconciler: start to sync state" Sep 08 02:21:44.881407 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.881324 2560 factory.go:55] Registering systemd factory Sep 08 02:21:44.881407 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.881341 2560 factory.go:223] Registration of the systemd container factory successfully Sep 08 02:21:44.881407 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:44.880199 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-33.ec2.internal.18d3372406a280b3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-33.ec2.internal,UID:ip-10-0-141-33.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-33.ec2.internal,},FirstTimestamp:2026-09-08 02:21:44.865882291 +0000 UTC m=+0.397527148,LastTimestamp:2026-09-08 02:21:44.865882291 +0000 UTC m=+0.397527148,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-33.ec2.internal,}" Sep 08 02:21:44.881611 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.881598 2560 factory.go:153] Registering CRI-O factory Sep 08 02:21:44.881682 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.881613 2560 factory.go:223] Registration of the crio container factory successfully Sep 08 02:21:44.881682 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.881658 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 08 02:21:44.881682 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.881678 2560 factory.go:103] Registering Raw factory Sep 08 02:21:44.881812 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.881690 2560 manager.go:1196] Started watching for new ooms in manager Sep 08 02:21:44.882491 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.882475 2560 manager.go:349] Starting recovery of all containers Sep 08 02:21:44.884435 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:44.884407 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 08 02:21:44.884663 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:44.884617 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-33.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 08 02:21:44.892106 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.891953 2560 manager.go:354] Recovery completed Sep 08 02:21:44.896926 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.896903 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 02:21:44.899103 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.899080 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientMemory" Sep 08 02:21:44.899172 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.899120 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 02:21:44.899172 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.899135 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientPID" Sep 08 02:21:44.899734 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.899716 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 08 02:21:44.899734 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.899730 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 08 02:21:44.899852 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.899749 2560 state_mem.go:36] "Initialized new in-memory state store" Sep 08 02:21:44.899852 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.899770 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 02:21:44.900581 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.900566 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientMemory" Sep 08 02:21:44.900653 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.900594 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 02:21:44.900653 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.900607 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientPID" Sep 08 02:21:44.900871 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.900859 2560 policy_none.go:49] "None policy: Start" Sep 08 02:21:44.900903 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.900878 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 08 02:21:44.900931 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.900912 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 08 02:21:44.901928 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.901916 2560 policy_none.go:47] "Start" Sep 08 02:21:44.933204 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.933185 2560 manager.go:337] "Starting Device Plugin manager" Sep 08 02:21:44.950434 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:44.933226 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 08 02:21:44.950434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.933257 2560 server.go:87] "Starting device plugin registration server" Sep 08 02:21:44.950434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.933680 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 08 02:21:44.950434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.933691 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 08 02:21:44.950434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.933783 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 08 02:21:44.950434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.933883 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 08 02:21:44.950434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:44.933891 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 08 02:21:44.950434 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:44.934532 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 08 02:21:44.950434 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:44.934598 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-33.ec2.internal\" not found" Sep 08 02:21:45.009294 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.009261 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 08 02:21:45.010531 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.010507 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 08 02:21:45.010648 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.010586 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 08 02:21:45.010648 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.010617 2560 kubelet.go:2448] "Starting kubelet main sync loop" Sep 08 02:21:45.010742 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:45.010726 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 08 02:21:45.012835 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.012815 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 02:21:45.034759 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.034649 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 02:21:45.035631 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.035613 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientMemory" Sep 08 02:21:45.035710 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.035649 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 02:21:45.035710 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.035663 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientPID" Sep 08 02:21:45.035710 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.035687 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-33.ec2.internal" Sep 08 02:21:45.052368 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.052345 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-33.ec2.internal" Sep 08 02:21:45.052478 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:45.052373 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-33.ec2.internal\": node \"ip-10-0-141-33.ec2.internal\" not found" Sep 08 02:21:45.081059 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:45.081021 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Sep 08 02:21:45.111279 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.111253 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-33.ec2.internal"] Sep 08 02:21:45.111367 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.111327 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 02:21:45.112794 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.112775 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientMemory" Sep 08 02:21:45.112890 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.112805 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 02:21:45.112890 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.112814 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientPID" Sep 08 02:21:45.115050 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.115036 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 02:21:45.115192 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.115176 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" Sep 08 02:21:45.115252 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.115213 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 02:21:45.115789 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.115773 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientMemory" Sep 08 02:21:45.115857 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.115797 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 02:21:45.115857 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.115807 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientPID" Sep 08 02:21:45.115857 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.115827 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientMemory" Sep 08 02:21:45.115857 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.115849 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 02:21:45.116033 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.115871 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientPID" Sep 08 02:21:45.118186 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.118171 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-33.ec2.internal" Sep 08 02:21:45.118236 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.118201 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 02:21:45.118879 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.118865 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientMemory" Sep 08 02:21:45.118936 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.118890 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 02:21:45.118936 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.118899 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientPID" Sep 08 02:21:45.138158 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:45.138136 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-33.ec2.internal\" not found" node="ip-10-0-141-33.ec2.internal" Sep 08 02:21:45.142623 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:45.142607 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-33.ec2.internal\" not found" node="ip-10-0-141-33.ec2.internal" Sep 08 02:21:45.182045 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:45.182018 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Sep 08 02:21:45.282904 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:45.282868 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Sep 08 02:21:45.282996 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.282928 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aa175197885ff6630fec12edf06719d3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal\" (UID: \"aa175197885ff6630fec12edf06719d3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" Sep 08 02:21:45.282996 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.282952 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa175197885ff6630fec12edf06719d3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal\" (UID: \"aa175197885ff6630fec12edf06719d3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" Sep 08 02:21:45.282996 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.282967 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a1d443509e8525aa8dff4d1a40896247-config\") pod \"kube-apiserver-proxy-ip-10-0-141-33.ec2.internal\" (UID: \"a1d443509e8525aa8dff4d1a40896247\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-33.ec2.internal" Sep 08 02:21:45.383569 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:45.383499 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Sep 08 02:21:45.383569 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.383536 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a1d443509e8525aa8dff4d1a40896247-config\") pod \"kube-apiserver-proxy-ip-10-0-141-33.ec2.internal\" (UID: \"a1d443509e8525aa8dff4d1a40896247\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-33.ec2.internal" Sep 08 02:21:45.383569 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.383572 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aa175197885ff6630fec12edf06719d3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal\" (UID: \"aa175197885ff6630fec12edf06719d3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" Sep 08 02:21:45.383682 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.383588 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa175197885ff6630fec12edf06719d3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal\" (UID: \"aa175197885ff6630fec12edf06719d3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" Sep 08 02:21:45.383682 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.383635 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa175197885ff6630fec12edf06719d3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal\" (UID: \"aa175197885ff6630fec12edf06719d3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" Sep 08 02:21:45.383682 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.383635 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aa175197885ff6630fec12edf06719d3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal\" (UID: \"aa175197885ff6630fec12edf06719d3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" Sep 08 02:21:45.383682 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.383655 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a1d443509e8525aa8dff4d1a40896247-config\") pod \"kube-apiserver-proxy-ip-10-0-141-33.ec2.internal\" (UID: \"a1d443509e8525aa8dff4d1a40896247\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-33.ec2.internal" Sep 08 02:21:45.440954 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.440919 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" Sep 08 02:21:45.444839 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.444809 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-33.ec2.internal" Sep 08 02:21:45.484159 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:45.484127 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Sep 08 02:21:45.585536 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:45.585461 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Sep 08 02:21:45.686371 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:45.686276 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Sep 08 02:21:45.787420 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:45.787378 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Sep 08 02:21:45.806128 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.806097 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 02:21:45.809978 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.809962 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 08 02:21:45.810137 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.810113 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 08 02:21:45.810178 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.810124 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 08 02:21:45.884002 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.883957 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-07 02:16:45 +0000 UTC" deadline="2028-04-04 11:49:09.955118699 +0000 UTC" Sep 08 02:21:45.884002 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:45.883991 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13785h27m24.071130321s" Sep 08 02:21:45.888357 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:45.888337 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Sep 08 02:21:45.988807 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:45.988775 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Sep 08 02:21:46.015936 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.015908 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 02:21:46.041446 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.041419 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 02:21:46.081459 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.081426 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" Sep 08 02:21:46.099994 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.099969 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 08 02:21:46.101155 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.101141 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-33.ec2.internal" Sep 08 02:21:46.112233 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.112212 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 08 02:21:46.133575 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:46.133526 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa175197885ff6630fec12edf06719d3.slice/crio-a5c8fd33f504638684991400290ac0302c244ffe6b397ea4088c53c0d9fc0680 WatchSource:0}: Error finding container a5c8fd33f504638684991400290ac0302c244ffe6b397ea4088c53c0d9fc0680: Status 404 returned error can't find the container with id a5c8fd33f504638684991400290ac0302c244ffe6b397ea4088c53c0d9fc0680 Sep 08 02:21:46.133980 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:46.133962 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1d443509e8525aa8dff4d1a40896247.slice/crio-5129087b66ccd3a10fe8e0e57d57fef5f9c80e21e982decd185ad68d9e699220 WatchSource:0}: Error finding container 5129087b66ccd3a10fe8e0e57d57fef5f9c80e21e982decd185ad68d9e699220: Status 404 returned error can't find the container with id 5129087b66ccd3a10fe8e0e57d57fef5f9c80e21e982decd185ad68d9e699220 Sep 08 02:21:46.142528 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.142509 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 02:21:46.852785 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.852749 2560 apiserver.go:52] "Watching apiserver" Sep 08 02:21:46.864765 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.864727 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 08 02:21:46.867271 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.867243 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-66vxb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal","openshift-multus/multus-252ld","openshift-network-diagnostics/network-check-target-c8fz9","openshift-cluster-node-tuning-operator/tuned-x6s55","openshift-image-registry/node-ca-vn7fg","openshift-multus/multus-additional-cni-plugins-8mcwm","openshift-multus/network-metrics-daemon-mt4gl","openshift-network-operator/iptables-alerter-8w8k7","openshift-ovn-kubernetes/ovnkube-node-wcjds","kube-system/global-pull-secret-syncer-pmw7v","kube-system/konnectivity-agent-gp9px","kube-system/kube-apiserver-proxy-ip-10-0-141-33.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq"] Sep 08 02:21:46.870065 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.870041 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8mcwm" Sep 08 02:21:46.872819 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.872794 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:21:46.872929 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:46.872881 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmw7v" podUID="0af6038b-8bad-4d6a-9478-083edfbc974a" Sep 08 02:21:46.873474 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.873433 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 08 02:21:46.873785 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.873767 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 08 02:21:46.873873 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.873784 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mp2nc\"" Sep 08 02:21:46.873936 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.873767 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 08 02:21:46.874443 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.874423 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 08 02:21:46.876033 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.876007 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 08 02:21:46.876129 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.876056 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-252ld" Sep 08 02:21:46.877848 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.877829 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:21:46.877951 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:46.877922 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c8fz9" podUID="fe2f6a1b-8c73-4c40-8525-d1329e889f46" Sep 08 02:21:46.879078 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.879058 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-cffq9\"" Sep 08 02:21:46.879276 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.879249 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 08 02:21:46.880266 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.880244 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:46.881984 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.881937 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 08 02:21:46.883409 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.882966 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vn7fg" Sep 08 02:21:46.883936 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.883639 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 08 02:21:46.883936 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.883682 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 08 02:21:46.884080 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.883952 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ss6rk\"" Sep 08 02:21:46.885991 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.885969 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 08 02:21:46.886214 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.886196 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 08 02:21:46.886301 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.886244 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-l2g59\"" Sep 08 02:21:46.886461 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.886444 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 08 02:21:46.886632 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.886613 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8w8k7" Sep 08 02:21:46.888491 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.888457 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" Sep 08 02:21:46.890415 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.889663 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 08 02:21:46.890415 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.890048 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-r7m6m\"" Sep 08 02:21:46.890415 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.890334 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 08 02:21:46.890691 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.890534 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 08 02:21:46.891212 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.891194 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 08 02:21:46.892066 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.891601 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jnfzt\"" Sep 08 02:21:46.892066 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.891848 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 08 02:21:46.892066 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.891870 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 08 02:21:46.892806 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.892484 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:21:46.892806 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:46.892582 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mt4gl" podUID="ec8f0aba-d0c2-4d45-b25d-84e2790d49ec" Sep 08 02:21:46.894946 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.894923 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-etc-modprobe-d\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:46.895045 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.894963 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0af6038b-8bad-4d6a-9478-083edfbc974a-dbus\") pod \"global-pull-secret-syncer-pmw7v\" (UID: \"0af6038b-8bad-4d6a-9478-083edfbc974a\") " pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:21:46.895045 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.894990 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-host-run-k8s-cni-cncf-io\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:46.895045 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.895017 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-host-run-netns\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:46.895045 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.895041 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-multus-conf-dir\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:46.895258 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.895064 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsv7v\" (UniqueName: \"kubernetes.io/projected/6fe0dc65-7890-405f-9c55-755957407695-kube-api-access-nsv7v\") pod \"node-ca-vn7fg\" (UID: \"6fe0dc65-7890-405f-9c55-755957407695\") " pod="openshift-image-registry/node-ca-vn7fg" Sep 08 02:21:46.895258 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.895090 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-host-var-lib-cni-multus\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:46.895258 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.895114 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-etc-sysconfig\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:46.895258 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.895142 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjmfl\" (UniqueName: \"kubernetes.io/projected/76080513-3119-49e3-b19d-8c1a0d6f854c-kube-api-access-cjmfl\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:46.895258 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.895166 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6fe0dc65-7890-405f-9c55-755957407695-serviceca\") pod \"node-ca-vn7fg\" (UID: \"6fe0dc65-7890-405f-9c55-755957407695\") " pod="openshift-image-registry/node-ca-vn7fg" Sep 08 02:21:46.895258 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.895194 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab6018f8-3012-4ec8-959e-95479604e898-cni-binary-copy\") pod \"multus-additional-cni-plugins-8mcwm\" (UID: \"ab6018f8-3012-4ec8-959e-95479604e898\") " pod="openshift-multus/multus-additional-cni-plugins-8mcwm" Sep 08 02:21:46.895559 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.895219 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/196615ff-9773-44ba-b5a7-a56229dafb40-cni-binary-copy\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:46.895559 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.895457 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-host-var-lib-kubelet\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:46.895559 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.895523 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-etc-sysctl-d\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:46.895705 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.895559 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-etc-sysctl-conf\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:46.895705 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.895592 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-etc-systemd\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:46.895796 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.895620 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-lib-modules\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:46.895852 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.895836 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret\") pod \"global-pull-secret-syncer-pmw7v\" (UID: \"0af6038b-8bad-4d6a-9478-083edfbc974a\") " pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:21:46.895908 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.895871 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-cnibin\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:46.895962 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.895903 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-hostroot\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:46.895962 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.895937 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-etc-kubernetes\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:46.896147 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.896125 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-run\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:46.896210 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.896161 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-sys\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:46.896210 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.896196 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab6018f8-3012-4ec8-959e-95479604e898-system-cni-dir\") pod \"multus-additional-cni-plugins-8mcwm\" (UID: \"ab6018f8-3012-4ec8-959e-95479604e898\") " pod="openshift-multus/multus-additional-cni-plugins-8mcwm" Sep 08 02:21:46.896315 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.896230 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab6018f8-3012-4ec8-959e-95479604e898-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8mcwm\" (UID: \"ab6018f8-3012-4ec8-959e-95479604e898\") " pod="openshift-multus/multus-additional-cni-plugins-8mcwm" Sep 08 02:21:46.896315 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.896264 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0af6038b-8bad-4d6a-9478-083edfbc974a-kubelet-config\") pod \"global-pull-secret-syncer-pmw7v\" (UID: \"0af6038b-8bad-4d6a-9478-083edfbc974a\") " pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:21:46.896495 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.896475 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-host-run-multus-certs\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:46.896579 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.896511 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6fe0dc65-7890-405f-9c55-755957407695-host\") pod \"node-ca-vn7fg\" (UID: \"6fe0dc65-7890-405f-9c55-755957407695\") " pod="openshift-image-registry/node-ca-vn7fg" Sep 08 02:21:46.896579 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.896557 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab6018f8-3012-4ec8-959e-95479604e898-cnibin\") pod \"multus-additional-cni-plugins-8mcwm\" (UID: \"ab6018f8-3012-4ec8-959e-95479604e898\") " pod="openshift-multus/multus-additional-cni-plugins-8mcwm" Sep 08 02:21:46.896686 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.896590 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-multus-socket-dir-parent\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:46.896686 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.896623 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-host-var-lib-cni-bin\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:46.897089 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.897067 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:46.897991 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.897958 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxvzz\" (UniqueName: \"kubernetes.io/projected/196615ff-9773-44ba-b5a7-a56229dafb40-kube-api-access-xxvzz\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:46.898178 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.898154 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2987\" (UniqueName: \"kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987\") pod \"network-check-target-c8fz9\" (UID: \"fe2f6a1b-8c73-4c40-8525-d1329e889f46\") " pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:21:46.898296 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.898282 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-host\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:46.898553 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.898426 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab6018f8-3012-4ec8-959e-95479604e898-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8mcwm\" (UID: \"ab6018f8-3012-4ec8-959e-95479604e898\") " pod="openshift-multus/multus-additional-cni-plugins-8mcwm" Sep 08 02:21:46.898735 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.898669 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhwgc\" (UniqueName: \"kubernetes.io/projected/ab6018f8-3012-4ec8-959e-95479604e898-kube-api-access-fhwgc\") pod \"multus-additional-cni-plugins-8mcwm\" (UID: \"ab6018f8-3012-4ec8-959e-95479604e898\") " pod="openshift-multus/multus-additional-cni-plugins-8mcwm" Sep 08 02:21:46.898843 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.898828 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-system-cni-dir\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:46.898951 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.898935 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-etc-kubernetes\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:46.899050 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.899037 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-var-lib-kubelet\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:46.899166 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.899142 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/76080513-3119-49e3-b19d-8c1a0d6f854c-etc-tuned\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:46.899250 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.899183 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76080513-3119-49e3-b19d-8c1a0d6f854c-tmp\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:46.899328 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.899220 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab6018f8-3012-4ec8-959e-95479604e898-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8mcwm\" (UID: \"ab6018f8-3012-4ec8-959e-95479604e898\") " pod="openshift-multus/multus-additional-cni-plugins-8mcwm" Sep 08 02:21:46.899370 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.899350 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-multus-cni-dir\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:46.899428 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.899395 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/196615ff-9773-44ba-b5a7-a56229dafb40-multus-daemon-config\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:46.901260 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.901237 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gzt98\"" Sep 08 02:21:46.901564 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.901527 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 08 02:21:46.903050 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.902659 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 08 02:21:46.903050 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.902764 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 08 02:21:46.903050 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.902802 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 08 02:21:46.903050 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.902764 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 08 02:21:46.903050 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.902962 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 08 02:21:46.903380 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.903275 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 08 02:21:46.903694 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.903673 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-66vxb" Sep 08 02:21:46.908111 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.906594 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gp9px" Sep 08 02:21:46.908111 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.907104 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 08 02:21:46.908111 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.907186 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4w24l\"" Sep 08 02:21:46.909316 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.908597 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 08 02:21:46.909425 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.909374 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 08 02:21:46.910071 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.910050 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 08 02:21:46.910269 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.910252 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vxcqb\"" Sep 08 02:21:46.922946 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.922919 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qvjbt" Sep 08 02:21:46.934768 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.934742 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qvjbt" Sep 08 02:21:46.982206 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:46.982176 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 08 02:21:47.000311 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000274 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-multus-cni-dir\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.000460 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000323 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bdf9207b-a3c4-4579-abdc-de9a2465f484-host-slash\") pod \"iptables-alerter-8w8k7\" (UID: \"bdf9207b-a3c4-4579-abdc-de9a2465f484\") " pod="openshift-network-operator/iptables-alerter-8w8k7" Sep 08 02:21:47.000460 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000380 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-host-run-netns\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.000460 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000426 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-multus-cni-dir\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.000460 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000449 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwpnh\" (UniqueName: \"kubernetes.io/projected/05f8db49-040c-460f-b931-90988ff34ae7-kube-api-access-rwpnh\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.000714 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000500 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b9a058d-34e6-4e9e-9623-afbc7e0fa776-device-dir\") pod \"aws-ebs-csi-driver-node-lzpmq\" (UID: \"6b9a058d-34e6-4e9e-9623-afbc7e0fa776\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" Sep 08 02:21:47.000714 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000537 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0af6038b-8bad-4d6a-9478-083edfbc974a-dbus\") pod \"global-pull-secret-syncer-pmw7v\" (UID: \"0af6038b-8bad-4d6a-9478-083edfbc974a\") " pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:21:47.000714 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000565 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsv7v\" (UniqueName: \"kubernetes.io/projected/6fe0dc65-7890-405f-9c55-755957407695-kube-api-access-nsv7v\") pod \"node-ca-vn7fg\" (UID: \"6fe0dc65-7890-405f-9c55-755957407695\") " pod="openshift-image-registry/node-ca-vn7fg" Sep 08 02:21:47.000714 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000592 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b9a058d-34e6-4e9e-9623-afbc7e0fa776-registration-dir\") pod \"aws-ebs-csi-driver-node-lzpmq\" (UID: \"6b9a058d-34e6-4e9e-9623-afbc7e0fa776\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" Sep 08 02:21:47.000714 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000619 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-host-var-lib-cni-multus\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.000714 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000650 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjmfl\" (UniqueName: \"kubernetes.io/projected/76080513-3119-49e3-b19d-8c1a0d6f854c-kube-api-access-cjmfl\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.000714 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000675 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6fe0dc65-7890-405f-9c55-755957407695-serviceca\") pod \"node-ca-vn7fg\" (UID: \"6fe0dc65-7890-405f-9c55-755957407695\") " pod="openshift-image-registry/node-ca-vn7fg" Sep 08 02:21:47.000714 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000701 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqt5h\" (UniqueName: \"kubernetes.io/projected/bdf9207b-a3c4-4579-abdc-de9a2465f484-kube-api-access-rqt5h\") pod \"iptables-alerter-8w8k7\" (UID: \"bdf9207b-a3c4-4579-abdc-de9a2465f484\") " pod="openshift-network-operator/iptables-alerter-8w8k7" Sep 08 02:21:47.001129 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000728 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-host-kubelet\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.001129 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000754 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/550f906c-4554-42d1-9ef7-1922f2501656-konnectivity-ca\") pod \"konnectivity-agent-gp9px\" (UID: \"550f906c-4554-42d1-9ef7-1922f2501656\") " pod="kube-system/konnectivity-agent-gp9px" Sep 08 02:21:47.001129 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000780 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/196615ff-9773-44ba-b5a7-a56229dafb40-cni-binary-copy\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.001129 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000803 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-host-var-lib-kubelet\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.001129 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000828 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-etc-sysctl-d\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.001129 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000850 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-etc-systemd\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.001129 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000870 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-lib-modules\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.001129 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000898 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svhn8\" (UniqueName: \"kubernetes.io/projected/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-kube-api-access-svhn8\") pod \"network-metrics-daemon-mt4gl\" (UID: \"ec8f0aba-d0c2-4d45-b25d-84e2790d49ec\") " pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:21:47.001129 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000928 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-systemd-units\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.001129 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000952 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b9a058d-34e6-4e9e-9623-afbc7e0fa776-etc-selinux\") pod \"aws-ebs-csi-driver-node-lzpmq\" (UID: \"6b9a058d-34e6-4e9e-9623-afbc7e0fa776\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" Sep 08 02:21:47.001129 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.000979 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret\") pod \"global-pull-secret-syncer-pmw7v\" (UID: \"0af6038b-8bad-4d6a-9478-083edfbc974a\") " pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:21:47.001129 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001005 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-cnibin\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.001129 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001026 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-host-slash\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.001129 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001048 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-log-socket\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.001129 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001072 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn8x8\" (UniqueName: \"kubernetes.io/projected/185c152f-08c1-48bf-9d40-f61ee106ff12-kube-api-access-dn8x8\") pod \"node-resolver-66vxb\" (UID: \"185c152f-08c1-48bf-9d40-f61ee106ff12\") " pod="openshift-dns/node-resolver-66vxb" Sep 08 02:21:47.001129 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001108 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b9a058d-34e6-4e9e-9623-afbc7e0fa776-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lzpmq\" (UID: \"6b9a058d-34e6-4e9e-9623-afbc7e0fa776\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" Sep 08 02:21:47.001833 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001157 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-host-run-multus-certs\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.001833 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001178 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0af6038b-8bad-4d6a-9478-083edfbc974a-dbus\") pod \"global-pull-secret-syncer-pmw7v\" (UID: \"0af6038b-8bad-4d6a-9478-083edfbc974a\") " pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:21:47.001833 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001190 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6fe0dc65-7890-405f-9c55-755957407695-host\") pod \"node-ca-vn7fg\" (UID: \"6fe0dc65-7890-405f-9c55-755957407695\") " pod="openshift-image-registry/node-ca-vn7fg" Sep 08 02:21:47.001833 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001215 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs\") pod \"network-metrics-daemon-mt4gl\" (UID: \"ec8f0aba-d0c2-4d45-b25d-84e2790d49ec\") " pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:21:47.001833 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001246 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/05f8db49-040c-460f-b931-90988ff34ae7-ovn-node-metrics-cert\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.001833 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001269 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/185c152f-08c1-48bf-9d40-f61ee106ff12-tmp-dir\") pod \"node-resolver-66vxb\" (UID: \"185c152f-08c1-48bf-9d40-f61ee106ff12\") " pod="openshift-dns/node-resolver-66vxb" Sep 08 02:21:47.001833 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001295 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-multus-socket-dir-parent\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.001833 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001320 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxvzz\" (UniqueName: \"kubernetes.io/projected/196615ff-9773-44ba-b5a7-a56229dafb40-kube-api-access-xxvzz\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.001833 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001345 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-host\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.001833 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001378 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-etc-kubernetes\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.001833 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001413 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-var-lib-kubelet\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.001833 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001442 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/76080513-3119-49e3-b19d-8c1a0d6f854c-etc-tuned\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.001833 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001465 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76080513-3119-49e3-b19d-8c1a0d6f854c-tmp\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.001833 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001491 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-host-cni-netd\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.001833 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001516 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/05f8db49-040c-460f-b931-90988ff34ae7-ovnkube-script-lib\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.001833 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001566 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab6018f8-3012-4ec8-959e-95479604e898-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8mcwm\" (UID: \"ab6018f8-3012-4ec8-959e-95479604e898\") " pod="openshift-multus/multus-additional-cni-plugins-8mcwm" Sep 08 02:21:47.001833 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001592 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/196615ff-9773-44ba-b5a7-a56229dafb40-multus-daemon-config\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.002378 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001628 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-etc-modprobe-d\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.002378 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001655 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-host-run-k8s-cni-cncf-io\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.002378 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001681 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-host-run-netns\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.002378 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001707 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-multus-conf-dir\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.002378 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001733 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bdf9207b-a3c4-4579-abdc-de9a2465f484-iptables-alerter-script\") pod \"iptables-alerter-8w8k7\" (UID: \"bdf9207b-a3c4-4579-abdc-de9a2465f484\") " pod="openshift-network-operator/iptables-alerter-8w8k7" Sep 08 02:21:47.002378 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001761 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-run-systemd\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.002378 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001792 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-var-lib-openvswitch\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.002378 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001780 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-host-run-multus-certs\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.002378 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001821 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-run-ovn\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.002378 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001849 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-etc-sysconfig\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.002378 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001868 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6fe0dc65-7890-405f-9c55-755957407695-host\") pod \"node-ca-vn7fg\" (UID: \"6fe0dc65-7890-405f-9c55-755957407695\") " pod="openshift-image-registry/node-ca-vn7fg" Sep 08 02:21:47.002378 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001897 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-etc-openvswitch\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.002378 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001921 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-node-log\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.002378 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001945 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-host-cni-bin\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.002378 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001960 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-multus-socket-dir-parent\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.002378 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.001973 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.002378 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002075 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-host-var-lib-cni-multus\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.003055 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002186 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/196615ff-9773-44ba-b5a7-a56229dafb40-cni-binary-copy\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.003055 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002226 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-host\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.003055 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002252 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/05f8db49-040c-460f-b931-90988ff34ae7-ovnkube-config\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.003055 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002279 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab6018f8-3012-4ec8-959e-95479604e898-cni-binary-copy\") pod \"multus-additional-cni-plugins-8mcwm\" (UID: \"ab6018f8-3012-4ec8-959e-95479604e898\") " pod="openshift-multus/multus-additional-cni-plugins-8mcwm" Sep 08 02:21:47.003055 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002281 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-host-var-lib-kubelet\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.003055 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002294 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-var-lib-kubelet\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.003055 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002251 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-etc-kubernetes\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.003055 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002382 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-cnibin\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.003055 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002409 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-etc-systemd\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.003055 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002440 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-host-run-k8s-cni-cncf-io\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.003055 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002474 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-lib-modules\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.003055 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002490 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-host-run-netns\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.003055 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002506 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-etc-sysctl-conf\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.003055 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002533 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-hostroot\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.003055 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002579 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-etc-kubernetes\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.003055 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002606 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-run\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.003055 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002629 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-sys\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.003055 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002656 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b9a058d-34e6-4e9e-9623-afbc7e0fa776-sys-fs\") pod \"aws-ebs-csi-driver-node-lzpmq\" (UID: \"6b9a058d-34e6-4e9e-9623-afbc7e0fa776\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" Sep 08 02:21:47.003750 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002716 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab6018f8-3012-4ec8-959e-95479604e898-system-cni-dir\") pod \"multus-additional-cni-plugins-8mcwm\" (UID: \"ab6018f8-3012-4ec8-959e-95479604e898\") " pod="openshift-multus/multus-additional-cni-plugins-8mcwm" Sep 08 02:21:47.003750 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002745 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab6018f8-3012-4ec8-959e-95479604e898-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8mcwm\" (UID: \"ab6018f8-3012-4ec8-959e-95479604e898\") " pod="openshift-multus/multus-additional-cni-plugins-8mcwm" Sep 08 02:21:47.003750 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002772 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0af6038b-8bad-4d6a-9478-083edfbc974a-kubelet-config\") pod \"global-pull-secret-syncer-pmw7v\" (UID: \"0af6038b-8bad-4d6a-9478-083edfbc974a\") " pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:21:47.003750 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002799 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/185c152f-08c1-48bf-9d40-f61ee106ff12-hosts-file\") pod \"node-resolver-66vxb\" (UID: \"185c152f-08c1-48bf-9d40-f61ee106ff12\") " pod="openshift-dns/node-resolver-66vxb" Sep 08 02:21:47.003750 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002828 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b9a058d-34e6-4e9e-9623-afbc7e0fa776-socket-dir\") pod \"aws-ebs-csi-driver-node-lzpmq\" (UID: \"6b9a058d-34e6-4e9e-9623-afbc7e0fa776\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" Sep 08 02:21:47.003750 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002859 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smsdx\" (UniqueName: \"kubernetes.io/projected/6b9a058d-34e6-4e9e-9623-afbc7e0fa776-kube-api-access-smsdx\") pod \"aws-ebs-csi-driver-node-lzpmq\" (UID: \"6b9a058d-34e6-4e9e-9623-afbc7e0fa776\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" Sep 08 02:21:47.003750 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002889 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab6018f8-3012-4ec8-959e-95479604e898-cnibin\") pod \"multus-additional-cni-plugins-8mcwm\" (UID: \"ab6018f8-3012-4ec8-959e-95479604e898\") " pod="openshift-multus/multus-additional-cni-plugins-8mcwm" Sep 08 02:21:47.003750 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.002916 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-host-var-lib-cni-bin\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.003750 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:47.002945 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 02:21:47.003750 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003022 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-etc-sysctl-d\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.003750 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003037 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2987\" (UniqueName: \"kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987\") pod \"network-check-target-c8fz9\" (UID: \"fe2f6a1b-8c73-4c40-8525-d1329e889f46\") " pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:21:47.003750 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003069 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-run-openvswitch\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.003750 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003097 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-host-run-ovn-kubernetes\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.003750 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003119 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-etc-modprobe-d\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.003750 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003130 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab6018f8-3012-4ec8-959e-95479604e898-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8mcwm\" (UID: \"ab6018f8-3012-4ec8-959e-95479604e898\") " pod="openshift-multus/multus-additional-cni-plugins-8mcwm" Sep 08 02:21:47.003750 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003142 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-etc-sysctl-conf\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.003750 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003160 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhwgc\" (UniqueName: \"kubernetes.io/projected/ab6018f8-3012-4ec8-959e-95479604e898-kube-api-access-fhwgc\") pod \"multus-additional-cni-plugins-8mcwm\" (UID: \"ab6018f8-3012-4ec8-959e-95479604e898\") " pod="openshift-multus/multus-additional-cni-plugins-8mcwm" Sep 08 02:21:47.004434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003178 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0af6038b-8bad-4d6a-9478-083edfbc974a-kubelet-config\") pod \"global-pull-secret-syncer-pmw7v\" (UID: \"0af6038b-8bad-4d6a-9478-083edfbc974a\") " pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:21:47.004434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003180 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-host-var-lib-cni-bin\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.004434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003189 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-system-cni-dir\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.004434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003177 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 08 02:21:47.004434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003228 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/05f8db49-040c-460f-b931-90988ff34ae7-env-overrides\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.004434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003236 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-system-cni-dir\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.004434 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:47.003259 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret podName:0af6038b-8bad-4d6a-9478-083edfbc974a nodeName:}" failed. No retries permitted until 2026-09-08 02:21:47.503236149 +0000 UTC m=+3.034881004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret") pod "global-pull-secret-syncer-pmw7v" (UID: "0af6038b-8bad-4d6a-9478-083edfbc974a") : object "kube-system"/"original-pull-secret" not registered Sep 08 02:21:47.004434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003266 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-hostroot\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.004434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003289 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab6018f8-3012-4ec8-959e-95479604e898-cnibin\") pod \"multus-additional-cni-plugins-8mcwm\" (UID: \"ab6018f8-3012-4ec8-959e-95479604e898\") " pod="openshift-multus/multus-additional-cni-plugins-8mcwm" Sep 08 02:21:47.004434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003301 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-multus-conf-dir\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.004434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003306 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/196615ff-9773-44ba-b5a7-a56229dafb40-etc-kubernetes\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.004434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003331 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/550f906c-4554-42d1-9ef7-1922f2501656-agent-certs\") pod \"konnectivity-agent-gp9px\" (UID: \"550f906c-4554-42d1-9ef7-1922f2501656\") " pod="kube-system/konnectivity-agent-gp9px" Sep 08 02:21:47.004434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003420 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-run\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.004434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003467 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-sys\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.004434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003671 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/196615ff-9773-44ba-b5a7-a56229dafb40-multus-daemon-config\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.004434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003874 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/76080513-3119-49e3-b19d-8c1a0d6f854c-etc-sysconfig\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.004434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003930 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab6018f8-3012-4ec8-959e-95479604e898-system-cni-dir\") pod \"multus-additional-cni-plugins-8mcwm\" (UID: \"ab6018f8-3012-4ec8-959e-95479604e898\") " pod="openshift-multus/multus-additional-cni-plugins-8mcwm" Sep 08 02:21:47.005278 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.003950 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6fe0dc65-7890-405f-9c55-755957407695-serviceca\") pod \"node-ca-vn7fg\" (UID: \"6fe0dc65-7890-405f-9c55-755957407695\") " pod="openshift-image-registry/node-ca-vn7fg" Sep 08 02:21:47.005278 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.004024 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab6018f8-3012-4ec8-959e-95479604e898-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8mcwm\" (UID: \"ab6018f8-3012-4ec8-959e-95479604e898\") " pod="openshift-multus/multus-additional-cni-plugins-8mcwm" Sep 08 02:21:47.005278 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.004170 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab6018f8-3012-4ec8-959e-95479604e898-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8mcwm\" (UID: \"ab6018f8-3012-4ec8-959e-95479604e898\") " pod="openshift-multus/multus-additional-cni-plugins-8mcwm" Sep 08 02:21:47.005278 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.004255 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab6018f8-3012-4ec8-959e-95479604e898-cni-binary-copy\") pod \"multus-additional-cni-plugins-8mcwm\" (UID: \"ab6018f8-3012-4ec8-959e-95479604e898\") " pod="openshift-multus/multus-additional-cni-plugins-8mcwm" Sep 08 02:21:47.005278 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.004374 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab6018f8-3012-4ec8-959e-95479604e898-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8mcwm\" (UID: \"ab6018f8-3012-4ec8-959e-95479604e898\") " pod="openshift-multus/multus-additional-cni-plugins-8mcwm" Sep 08 02:21:47.011205 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.007495 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76080513-3119-49e3-b19d-8c1a0d6f854c-tmp\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.013502 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.013424 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/76080513-3119-49e3-b19d-8c1a0d6f854c-etc-tuned\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.014994 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.014936 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjmfl\" (UniqueName: \"kubernetes.io/projected/76080513-3119-49e3-b19d-8c1a0d6f854c-kube-api-access-cjmfl\") pod \"tuned-x6s55\" (UID: \"76080513-3119-49e3-b19d-8c1a0d6f854c\") " pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.015360 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:47.015337 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 02:21:47.015452 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:47.015367 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 02:21:47.015452 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:47.015383 2560 projected.go:196] Error preparing data for projected volume kube-api-access-w2987 for pod openshift-network-diagnostics/network-check-target-c8fz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 02:21:47.015589 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:47.015451 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987 podName:fe2f6a1b-8c73-4c40-8525-d1329e889f46 nodeName:}" failed. No retries permitted until 2026-09-08 02:21:47.515424062 +0000 UTC m=+3.047068915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-w2987" (UniqueName: "kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987") pod "network-check-target-c8fz9" (UID: "fe2f6a1b-8c73-4c40-8525-d1329e889f46") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 02:21:47.019599 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.019553 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxvzz\" (UniqueName: \"kubernetes.io/projected/196615ff-9773-44ba-b5a7-a56229dafb40-kube-api-access-xxvzz\") pod \"multus-252ld\" (UID: \"196615ff-9773-44ba-b5a7-a56229dafb40\") " pod="openshift-multus/multus-252ld" Sep 08 02:21:47.020185 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.019814 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsv7v\" (UniqueName: \"kubernetes.io/projected/6fe0dc65-7890-405f-9c55-755957407695-kube-api-access-nsv7v\") pod \"node-ca-vn7fg\" (UID: \"6fe0dc65-7890-405f-9c55-755957407695\") " pod="openshift-image-registry/node-ca-vn7fg" Sep 08 02:21:47.020185 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.020059 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhwgc\" (UniqueName: \"kubernetes.io/projected/ab6018f8-3012-4ec8-959e-95479604e898-kube-api-access-fhwgc\") pod \"multus-additional-cni-plugins-8mcwm\" (UID: \"ab6018f8-3012-4ec8-959e-95479604e898\") " pod="openshift-multus/multus-additional-cni-plugins-8mcwm" Sep 08 02:21:47.020330 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.020271 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" event={"ID":"aa175197885ff6630fec12edf06719d3","Type":"ContainerStarted","Data":"a5c8fd33f504638684991400290ac0302c244ffe6b397ea4088c53c0d9fc0680"} Sep 08 02:21:47.025428 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.023420 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-33.ec2.internal" event={"ID":"a1d443509e8525aa8dff4d1a40896247","Type":"ContainerStarted","Data":"5129087b66ccd3a10fe8e0e57d57fef5f9c80e21e982decd185ad68d9e699220"} Sep 08 02:21:47.104581 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104117 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bdf9207b-a3c4-4579-abdc-de9a2465f484-iptables-alerter-script\") pod \"iptables-alerter-8w8k7\" (UID: \"bdf9207b-a3c4-4579-abdc-de9a2465f484\") " pod="openshift-network-operator/iptables-alerter-8w8k7" Sep 08 02:21:47.104581 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104168 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-run-systemd\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.104581 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104219 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-var-lib-openvswitch\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.104581 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104247 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-run-ovn\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.104581 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104274 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-etc-openvswitch\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.104581 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104298 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-node-log\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.104581 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104321 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-host-cni-bin\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.104581 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104348 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.104581 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104376 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/05f8db49-040c-460f-b931-90988ff34ae7-ovnkube-config\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.104581 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104407 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b9a058d-34e6-4e9e-9623-afbc7e0fa776-sys-fs\") pod \"aws-ebs-csi-driver-node-lzpmq\" (UID: \"6b9a058d-34e6-4e9e-9623-afbc7e0fa776\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" Sep 08 02:21:47.104581 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104415 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-run-systemd\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.104581 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104436 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/185c152f-08c1-48bf-9d40-f61ee106ff12-hosts-file\") pod \"node-resolver-66vxb\" (UID: \"185c152f-08c1-48bf-9d40-f61ee106ff12\") " pod="openshift-dns/node-resolver-66vxb" Sep 08 02:21:47.104581 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104459 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.104581 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104494 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b9a058d-34e6-4e9e-9623-afbc7e0fa776-socket-dir\") pod \"aws-ebs-csi-driver-node-lzpmq\" (UID: \"6b9a058d-34e6-4e9e-9623-afbc7e0fa776\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" Sep 08 02:21:47.104581 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104506 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/185c152f-08c1-48bf-9d40-f61ee106ff12-hosts-file\") pod \"node-resolver-66vxb\" (UID: \"185c152f-08c1-48bf-9d40-f61ee106ff12\") " pod="openshift-dns/node-resolver-66vxb" Sep 08 02:21:47.104581 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104526 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smsdx\" (UniqueName: \"kubernetes.io/projected/6b9a058d-34e6-4e9e-9623-afbc7e0fa776-kube-api-access-smsdx\") pod \"aws-ebs-csi-driver-node-lzpmq\" (UID: \"6b9a058d-34e6-4e9e-9623-afbc7e0fa776\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" Sep 08 02:21:47.104581 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104529 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-host-cni-bin\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.105629 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104415 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-etc-openvswitch\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.105629 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104572 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-var-lib-openvswitch\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.105629 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104578 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-run-ovn\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.105629 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104614 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-node-log\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.105629 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104682 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b9a058d-34e6-4e9e-9623-afbc7e0fa776-sys-fs\") pod \"aws-ebs-csi-driver-node-lzpmq\" (UID: \"6b9a058d-34e6-4e9e-9623-afbc7e0fa776\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" Sep 08 02:21:47.105629 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104806 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b9a058d-34e6-4e9e-9623-afbc7e0fa776-socket-dir\") pod \"aws-ebs-csi-driver-node-lzpmq\" (UID: \"6b9a058d-34e6-4e9e-9623-afbc7e0fa776\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" Sep 08 02:21:47.105629 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104810 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-run-openvswitch\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.105629 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104848 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-host-run-ovn-kubernetes\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.105629 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104853 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-run-openvswitch\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.105629 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104891 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/05f8db49-040c-460f-b931-90988ff34ae7-env-overrides\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.105629 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104896 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-host-run-ovn-kubernetes\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.105629 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.104928 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/550f906c-4554-42d1-9ef7-1922f2501656-agent-certs\") pod \"konnectivity-agent-gp9px\" (UID: \"550f906c-4554-42d1-9ef7-1922f2501656\") " pod="kube-system/konnectivity-agent-gp9px" Sep 08 02:21:47.105629 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105166 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/05f8db49-040c-460f-b931-90988ff34ae7-ovnkube-config\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.105629 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105221 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bdf9207b-a3c4-4579-abdc-de9a2465f484-host-slash\") pod \"iptables-alerter-8w8k7\" (UID: \"bdf9207b-a3c4-4579-abdc-de9a2465f484\") " pod="openshift-network-operator/iptables-alerter-8w8k7" Sep 08 02:21:47.105629 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105235 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/05f8db49-040c-460f-b931-90988ff34ae7-env-overrides\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.105629 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105247 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-host-run-netns\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.105629 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105281 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-host-run-netns\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.106618 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105300 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwpnh\" (UniqueName: \"kubernetes.io/projected/05f8db49-040c-460f-b931-90988ff34ae7-kube-api-access-rwpnh\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.106618 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105320 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bdf9207b-a3c4-4579-abdc-de9a2465f484-host-slash\") pod \"iptables-alerter-8w8k7\" (UID: \"bdf9207b-a3c4-4579-abdc-de9a2465f484\") " pod="openshift-network-operator/iptables-alerter-8w8k7" Sep 08 02:21:47.106618 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105329 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b9a058d-34e6-4e9e-9623-afbc7e0fa776-device-dir\") pod \"aws-ebs-csi-driver-node-lzpmq\" (UID: \"6b9a058d-34e6-4e9e-9623-afbc7e0fa776\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" Sep 08 02:21:47.106618 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105360 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b9a058d-34e6-4e9e-9623-afbc7e0fa776-registration-dir\") pod \"aws-ebs-csi-driver-node-lzpmq\" (UID: \"6b9a058d-34e6-4e9e-9623-afbc7e0fa776\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" Sep 08 02:21:47.106618 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105390 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqt5h\" (UniqueName: \"kubernetes.io/projected/bdf9207b-a3c4-4579-abdc-de9a2465f484-kube-api-access-rqt5h\") pod \"iptables-alerter-8w8k7\" (UID: \"bdf9207b-a3c4-4579-abdc-de9a2465f484\") " pod="openshift-network-operator/iptables-alerter-8w8k7" Sep 08 02:21:47.106618 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105415 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-host-kubelet\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.106618 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105423 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b9a058d-34e6-4e9e-9623-afbc7e0fa776-device-dir\") pod \"aws-ebs-csi-driver-node-lzpmq\" (UID: \"6b9a058d-34e6-4e9e-9623-afbc7e0fa776\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" Sep 08 02:21:47.106618 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105441 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/550f906c-4554-42d1-9ef7-1922f2501656-konnectivity-ca\") pod \"konnectivity-agent-gp9px\" (UID: \"550f906c-4554-42d1-9ef7-1922f2501656\") " pod="kube-system/konnectivity-agent-gp9px" Sep 08 02:21:47.106618 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105472 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svhn8\" (UniqueName: \"kubernetes.io/projected/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-kube-api-access-svhn8\") pod \"network-metrics-daemon-mt4gl\" (UID: \"ec8f0aba-d0c2-4d45-b25d-84e2790d49ec\") " pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:21:47.106618 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105498 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-systemd-units\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.106618 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105522 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b9a058d-34e6-4e9e-9623-afbc7e0fa776-etc-selinux\") pod \"aws-ebs-csi-driver-node-lzpmq\" (UID: \"6b9a058d-34e6-4e9e-9623-afbc7e0fa776\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" Sep 08 02:21:47.106618 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105611 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b9a058d-34e6-4e9e-9623-afbc7e0fa776-registration-dir\") pod \"aws-ebs-csi-driver-node-lzpmq\" (UID: \"6b9a058d-34e6-4e9e-9623-afbc7e0fa776\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" Sep 08 02:21:47.106618 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105656 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-host-slash\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.106618 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105671 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b9a058d-34e6-4e9e-9623-afbc7e0fa776-etc-selinux\") pod \"aws-ebs-csi-driver-node-lzpmq\" (UID: \"6b9a058d-34e6-4e9e-9623-afbc7e0fa776\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" Sep 08 02:21:47.106618 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105690 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-log-socket\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.106618 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105717 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn8x8\" (UniqueName: \"kubernetes.io/projected/185c152f-08c1-48bf-9d40-f61ee106ff12-kube-api-access-dn8x8\") pod \"node-resolver-66vxb\" (UID: \"185c152f-08c1-48bf-9d40-f61ee106ff12\") " pod="openshift-dns/node-resolver-66vxb" Sep 08 02:21:47.106618 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105742 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b9a058d-34e6-4e9e-9623-afbc7e0fa776-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lzpmq\" (UID: \"6b9a058d-34e6-4e9e-9623-afbc7e0fa776\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" Sep 08 02:21:47.107496 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105772 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs\") pod \"network-metrics-daemon-mt4gl\" (UID: \"ec8f0aba-d0c2-4d45-b25d-84e2790d49ec\") " pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:21:47.107496 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105797 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/05f8db49-040c-460f-b931-90988ff34ae7-ovn-node-metrics-cert\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.107496 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105821 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/185c152f-08c1-48bf-9d40-f61ee106ff12-tmp-dir\") pod \"node-resolver-66vxb\" (UID: \"185c152f-08c1-48bf-9d40-f61ee106ff12\") " pod="openshift-dns/node-resolver-66vxb" Sep 08 02:21:47.107496 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105853 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-host-cni-netd\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.107496 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105878 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/05f8db49-040c-460f-b931-90988ff34ae7-ovnkube-script-lib\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.107496 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105922 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-systemd-units\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.107496 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105974 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b9a058d-34e6-4e9e-9623-afbc7e0fa776-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lzpmq\" (UID: \"6b9a058d-34e6-4e9e-9623-afbc7e0fa776\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" Sep 08 02:21:47.107496 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.105999 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-host-slash\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.107496 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.106021 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-log-socket\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.107496 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.106145 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/550f906c-4554-42d1-9ef7-1922f2501656-konnectivity-ca\") pod \"konnectivity-agent-gp9px\" (UID: \"550f906c-4554-42d1-9ef7-1922f2501656\") " pod="kube-system/konnectivity-agent-gp9px" Sep 08 02:21:47.107496 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.106200 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-host-kubelet\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.107496 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.106441 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/05f8db49-040c-460f-b931-90988ff34ae7-ovnkube-script-lib\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.107496 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.106467 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/185c152f-08c1-48bf-9d40-f61ee106ff12-tmp-dir\") pod \"node-resolver-66vxb\" (UID: \"185c152f-08c1-48bf-9d40-f61ee106ff12\") " pod="openshift-dns/node-resolver-66vxb" Sep 08 02:21:47.107496 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.106493 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/05f8db49-040c-460f-b931-90988ff34ae7-host-cni-netd\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.107496 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:47.106605 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 02:21:47.107496 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:47.106683 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs podName:ec8f0aba-d0c2-4d45-b25d-84e2790d49ec nodeName:}" failed. No retries permitted until 2026-09-08 02:21:47.606652002 +0000 UTC m=+3.138296870 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs") pod "network-metrics-daemon-mt4gl" (UID: "ec8f0aba-d0c2-4d45-b25d-84e2790d49ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 02:21:47.107496 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.106845 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bdf9207b-a3c4-4579-abdc-de9a2465f484-iptables-alerter-script\") pod \"iptables-alerter-8w8k7\" (UID: \"bdf9207b-a3c4-4579-abdc-de9a2465f484\") " pod="openshift-network-operator/iptables-alerter-8w8k7" Sep 08 02:21:47.109046 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.109023 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/05f8db49-040c-460f-b931-90988ff34ae7-ovn-node-metrics-cert\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.109203 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.109136 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/550f906c-4554-42d1-9ef7-1922f2501656-agent-certs\") pod \"konnectivity-agent-gp9px\" (UID: \"550f906c-4554-42d1-9ef7-1922f2501656\") " pod="kube-system/konnectivity-agent-gp9px" Sep 08 02:21:47.116063 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.116040 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smsdx\" (UniqueName: \"kubernetes.io/projected/6b9a058d-34e6-4e9e-9623-afbc7e0fa776-kube-api-access-smsdx\") pod \"aws-ebs-csi-driver-node-lzpmq\" (UID: \"6b9a058d-34e6-4e9e-9623-afbc7e0fa776\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" Sep 08 02:21:47.120990 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.120963 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svhn8\" (UniqueName: \"kubernetes.io/projected/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-kube-api-access-svhn8\") pod \"network-metrics-daemon-mt4gl\" (UID: \"ec8f0aba-d0c2-4d45-b25d-84e2790d49ec\") " pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:21:47.121540 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.121519 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwpnh\" (UniqueName: \"kubernetes.io/projected/05f8db49-040c-460f-b931-90988ff34ae7-kube-api-access-rwpnh\") pod \"ovnkube-node-wcjds\" (UID: \"05f8db49-040c-460f-b931-90988ff34ae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.122067 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.122031 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqt5h\" (UniqueName: \"kubernetes.io/projected/bdf9207b-a3c4-4579-abdc-de9a2465f484-kube-api-access-rqt5h\") pod \"iptables-alerter-8w8k7\" (UID: \"bdf9207b-a3c4-4579-abdc-de9a2465f484\") " pod="openshift-network-operator/iptables-alerter-8w8k7" Sep 08 02:21:47.122151 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.122074 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn8x8\" (UniqueName: \"kubernetes.io/projected/185c152f-08c1-48bf-9d40-f61ee106ff12-kube-api-access-dn8x8\") pod \"node-resolver-66vxb\" (UID: \"185c152f-08c1-48bf-9d40-f61ee106ff12\") " pod="openshift-dns/node-resolver-66vxb" Sep 08 02:21:47.188672 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.188632 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8mcwm" Sep 08 02:21:47.199688 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.199660 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-252ld" Sep 08 02:21:47.215986 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.215958 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x6s55" Sep 08 02:21:47.222946 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.222917 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vn7fg" Sep 08 02:21:47.233634 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.233613 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8w8k7" Sep 08 02:21:47.251394 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.251370 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" Sep 08 02:21:47.260612 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.260579 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:21:47.268700 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.268678 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-66vxb" Sep 08 02:21:47.275412 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.275395 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gp9px" Sep 08 02:21:47.360570 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.360484 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 02:21:47.510088 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.510049 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret\") pod \"global-pull-secret-syncer-pmw7v\" (UID: \"0af6038b-8bad-4d6a-9478-083edfbc974a\") " pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:21:47.510277 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:47.510238 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 02:21:47.510339 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:47.510327 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret podName:0af6038b-8bad-4d6a-9478-083edfbc974a nodeName:}" failed. No retries permitted until 2026-09-08 02:21:48.510309027 +0000 UTC m=+4.041953886 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret") pod "global-pull-secret-syncer-pmw7v" (UID: "0af6038b-8bad-4d6a-9478-083edfbc974a") : object "kube-system"/"original-pull-secret" not registered Sep 08 02:21:47.579164 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.579126 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 02:21:47.610548 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.610508 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2987\" (UniqueName: \"kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987\") pod \"network-check-target-c8fz9\" (UID: \"fe2f6a1b-8c73-4c40-8525-d1329e889f46\") " pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:21:47.610780 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.610583 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs\") pod \"network-metrics-daemon-mt4gl\" (UID: \"ec8f0aba-d0c2-4d45-b25d-84e2790d49ec\") " pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:21:47.610780 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:47.610704 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 02:21:47.610780 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:47.610709 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 02:21:47.610780 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:47.610734 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 02:21:47.610780 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:47.610747 2560 projected.go:196] Error preparing data for projected volume kube-api-access-w2987 for pod openshift-network-diagnostics/network-check-target-c8fz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 02:21:47.610780 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:47.610770 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs podName:ec8f0aba-d0c2-4d45-b25d-84e2790d49ec nodeName:}" failed. No retries permitted until 2026-09-08 02:21:48.610749571 +0000 UTC m=+4.142394418 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs") pod "network-metrics-daemon-mt4gl" (UID: "ec8f0aba-d0c2-4d45-b25d-84e2790d49ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 02:21:47.611072 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:47.610802 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987 podName:fe2f6a1b-8c73-4c40-8525-d1329e889f46 nodeName:}" failed. No retries permitted until 2026-09-08 02:21:48.61078448 +0000 UTC m=+4.142429325 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-w2987" (UniqueName: "kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987") pod "network-check-target-c8fz9" (UID: "fe2f6a1b-8c73-4c40-8525-d1329e889f46") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 02:21:47.938148 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.938065 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 02:16:47 +0000 UTC" deadline="2028-02-10 14:40:47.524747471 +0000 UTC" Sep 08 02:21:47.938148 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:47.938097 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12492h18m59.586653755s" Sep 08 02:21:48.233034 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:48.232861 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab6018f8_3012_4ec8_959e_95479604e898.slice/crio-478494c83a8664064a4bde0e6dd906deb6f6e727f645f95d3125cfe74f9943eb WatchSource:0}: Error finding container 478494c83a8664064a4bde0e6dd906deb6f6e727f645f95d3125cfe74f9943eb: Status 404 returned error can't find the container with id 478494c83a8664064a4bde0e6dd906deb6f6e727f645f95d3125cfe74f9943eb Sep 08 02:21:48.253398 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:48.253364 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod185c152f_08c1_48bf_9d40_f61ee106ff12.slice/crio-88d27fed824154741b2a192ef24a37a6ff663809840593b20d5c466ddf511a5f WatchSource:0}: Error finding container 88d27fed824154741b2a192ef24a37a6ff663809840593b20d5c466ddf511a5f: Status 404 returned error can't find the container with id 88d27fed824154741b2a192ef24a37a6ff663809840593b20d5c466ddf511a5f Sep 08 02:21:48.254715 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:48.254681 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fe0dc65_7890_405f_9c55_755957407695.slice/crio-d46bf06cf2e029849568f86050cc94b839f1627e9231e55b9dd1e635ceeeb8b4 WatchSource:0}: Error finding container d46bf06cf2e029849568f86050cc94b839f1627e9231e55b9dd1e635ceeeb8b4: Status 404 returned error can't find the container with id d46bf06cf2e029849568f86050cc94b839f1627e9231e55b9dd1e635ceeeb8b4 Sep 08 02:21:48.255476 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:48.255450 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod196615ff_9773_44ba_b5a7_a56229dafb40.slice/crio-29978a0b0b5202b23a0d1600c231b7c6b779b8597857e9f080a94ab135521c72 WatchSource:0}: Error finding container 29978a0b0b5202b23a0d1600c231b7c6b779b8597857e9f080a94ab135521c72: Status 404 returned error can't find the container with id 29978a0b0b5202b23a0d1600c231b7c6b779b8597857e9f080a94ab135521c72 Sep 08 02:21:48.264166 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:48.264141 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05f8db49_040c_460f_b931_90988ff34ae7.slice/crio-58d27895cb3cdc0771d38020e7455bb3f2d48424ef63c9ea8ee9f5e20ba4f10d WatchSource:0}: Error finding container 58d27895cb3cdc0771d38020e7455bb3f2d48424ef63c9ea8ee9f5e20ba4f10d: Status 404 returned error can't find the container with id 58d27895cb3cdc0771d38020e7455bb3f2d48424ef63c9ea8ee9f5e20ba4f10d Sep 08 02:21:48.265491 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:21:48.265468 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdf9207b_a3c4_4579_abdc_de9a2465f484.slice/crio-3deaf8f2cd7f17b50192032e1b04b2574ed8501e32b866b3c8f72c61426885a7 WatchSource:0}: Error finding container 3deaf8f2cd7f17b50192032e1b04b2574ed8501e32b866b3c8f72c61426885a7: Status 404 returned error can't find the container with id 3deaf8f2cd7f17b50192032e1b04b2574ed8501e32b866b3c8f72c61426885a7 Sep 08 02:21:48.517111 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:48.517080 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret\") pod \"global-pull-secret-syncer-pmw7v\" (UID: \"0af6038b-8bad-4d6a-9478-083edfbc974a\") " pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:21:48.517252 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:48.517236 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 02:21:48.517320 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:48.517302 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret podName:0af6038b-8bad-4d6a-9478-083edfbc974a nodeName:}" failed. No retries permitted until 2026-09-08 02:21:50.517285613 +0000 UTC m=+6.048930462 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret") pod "global-pull-secret-syncer-pmw7v" (UID: "0af6038b-8bad-4d6a-9478-083edfbc974a") : object "kube-system"/"original-pull-secret" not registered Sep 08 02:21:48.618280 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:48.618241 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2987\" (UniqueName: \"kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987\") pod \"network-check-target-c8fz9\" (UID: \"fe2f6a1b-8c73-4c40-8525-d1329e889f46\") " pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:21:48.618460 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:48.618315 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs\") pod \"network-metrics-daemon-mt4gl\" (UID: \"ec8f0aba-d0c2-4d45-b25d-84e2790d49ec\") " pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:21:48.618460 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:48.618405 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 02:21:48.618460 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:48.618418 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 02:21:48.618460 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:48.618448 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 02:21:48.618460 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:48.618461 2560 projected.go:196] Error preparing data for projected volume kube-api-access-w2987 for pod openshift-network-diagnostics/network-check-target-c8fz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 02:21:48.618460 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:48.618466 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs podName:ec8f0aba-d0c2-4d45-b25d-84e2790d49ec nodeName:}" failed. No retries permitted until 2026-09-08 02:21:50.618448788 +0000 UTC m=+6.150093639 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs") pod "network-metrics-daemon-mt4gl" (UID: "ec8f0aba-d0c2-4d45-b25d-84e2790d49ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 02:21:48.618711 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:48.618534 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987 podName:fe2f6a1b-8c73-4c40-8525-d1329e889f46 nodeName:}" failed. No retries permitted until 2026-09-08 02:21:50.618517502 +0000 UTC m=+6.150162351 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-w2987" (UniqueName: "kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987") pod "network-check-target-c8fz9" (UID: "fe2f6a1b-8c73-4c40-8525-d1329e889f46") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 02:21:48.938851 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:48.938564 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 02:16:47 +0000 UTC" deadline="2028-06-21 02:36:49.3532346 +0000 UTC" Sep 08 02:21:48.938851 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:48.938718 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15648h15m0.414522428s" Sep 08 02:21:49.015081 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:49.013833 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:21:49.015081 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:49.013972 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmw7v" podUID="0af6038b-8bad-4d6a-9478-083edfbc974a" Sep 08 02:21:49.015081 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:49.014763 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:21:49.015081 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:49.014861 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c8fz9" podUID="fe2f6a1b-8c73-4c40-8525-d1329e889f46" Sep 08 02:21:49.015081 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:49.014940 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:21:49.015081 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:49.015026 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mt4gl" podUID="ec8f0aba-d0c2-4d45-b25d-84e2790d49ec" Sep 08 02:21:49.053059 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:49.052973 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8w8k7" event={"ID":"bdf9207b-a3c4-4579-abdc-de9a2465f484","Type":"ContainerStarted","Data":"3deaf8f2cd7f17b50192032e1b04b2574ed8501e32b866b3c8f72c61426885a7"} Sep 08 02:21:49.063602 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:49.063510 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-252ld" event={"ID":"196615ff-9773-44ba-b5a7-a56229dafb40","Type":"ContainerStarted","Data":"29978a0b0b5202b23a0d1600c231b7c6b779b8597857e9f080a94ab135521c72"} Sep 08 02:21:49.071332 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:49.071254 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vn7fg" event={"ID":"6fe0dc65-7890-405f-9c55-755957407695","Type":"ContainerStarted","Data":"d46bf06cf2e029849568f86050cc94b839f1627e9231e55b9dd1e635ceeeb8b4"} Sep 08 02:21:49.073572 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:49.073489 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-66vxb" event={"ID":"185c152f-08c1-48bf-9d40-f61ee106ff12","Type":"ContainerStarted","Data":"88d27fed824154741b2a192ef24a37a6ff663809840593b20d5c466ddf511a5f"} Sep 08 02:21:49.075524 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:49.075449 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8mcwm" event={"ID":"ab6018f8-3012-4ec8-959e-95479604e898","Type":"ContainerStarted","Data":"478494c83a8664064a4bde0e6dd906deb6f6e727f645f95d3125cfe74f9943eb"} Sep 08 02:21:49.080548 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:49.080363 2560 generic.go:356] "Generic (PLEG): container finished" podID="aa175197885ff6630fec12edf06719d3" containerID="10cafe1d24f26b792f75a6cb3c183389e221763cd93a6d10e4f0e4906658c40b" exitCode=0 Sep 08 02:21:49.080548 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:49.080452 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" event={"ID":"aa175197885ff6630fec12edf06719d3","Type":"ContainerDied","Data":"10cafe1d24f26b792f75a6cb3c183389e221763cd93a6d10e4f0e4906658c40b"} Sep 08 02:21:49.086559 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:49.085925 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x6s55" event={"ID":"76080513-3119-49e3-b19d-8c1a0d6f854c","Type":"ContainerStarted","Data":"6e05a5cb64b333632fb93b05271dee2cb0d63f15282019a611316ab5b3cd4557"} Sep 08 02:21:49.090445 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:49.090397 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" event={"ID":"05f8db49-040c-460f-b931-90988ff34ae7","Type":"ContainerStarted","Data":"58d27895cb3cdc0771d38020e7455bb3f2d48424ef63c9ea8ee9f5e20ba4f10d"} Sep 08 02:21:49.104356 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:49.104321 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-33.ec2.internal" event={"ID":"a1d443509e8525aa8dff4d1a40896247","Type":"ContainerStarted","Data":"3aff44c303a67d0c27ed8fcc044a952ee27cea3defe8a9b490d469fae07ddd1f"} Sep 08 02:21:49.128693 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:49.128002 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-33.ec2.internal" podStartSLOduration=3.127978649 podStartE2EDuration="3.127978649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 02:21:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:21:49.127162242 +0000 UTC m=+4.658807112" watchObservedRunningTime="2026-09-08 02:21:49.127978649 +0000 UTC m=+4.659623518" Sep 08 02:21:49.129157 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:49.129054 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gp9px" event={"ID":"550f906c-4554-42d1-9ef7-1922f2501656","Type":"ContainerStarted","Data":"5b75050d7f89cae283ec0a60e5724f1b35fef4f6581485d3bf062979aa415724"} Sep 08 02:21:49.144054 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:49.143952 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" event={"ID":"6b9a058d-34e6-4e9e-9623-afbc7e0fa776","Type":"ContainerStarted","Data":"9ca13c50b4e2673b3639e2322240f5fdeef72296ba50010cba54c542cbb3b1c9"} Sep 08 02:21:50.167600 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:50.167496 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" event={"ID":"aa175197885ff6630fec12edf06719d3","Type":"ContainerStarted","Data":"7101cfe6d787605a981cfde480c55f4ef8d23e755b831cee7c6e741a7fb733ac"} Sep 08 02:21:50.543868 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:50.543776 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret\") pod \"global-pull-secret-syncer-pmw7v\" (UID: \"0af6038b-8bad-4d6a-9478-083edfbc974a\") " pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:21:50.544033 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:50.543949 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 02:21:50.544033 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:50.544017 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret podName:0af6038b-8bad-4d6a-9478-083edfbc974a nodeName:}" failed. No retries permitted until 2026-09-08 02:21:54.543998491 +0000 UTC m=+10.075643335 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret") pod "global-pull-secret-syncer-pmw7v" (UID: "0af6038b-8bad-4d6a-9478-083edfbc974a") : object "kube-system"/"original-pull-secret" not registered Sep 08 02:21:50.644922 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:50.644880 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2987\" (UniqueName: \"kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987\") pod \"network-check-target-c8fz9\" (UID: \"fe2f6a1b-8c73-4c40-8525-d1329e889f46\") " pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:21:50.645113 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:50.644971 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs\") pod \"network-metrics-daemon-mt4gl\" (UID: \"ec8f0aba-d0c2-4d45-b25d-84e2790d49ec\") " pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:21:50.645183 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:50.645111 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 02:21:50.645183 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:50.645179 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs podName:ec8f0aba-d0c2-4d45-b25d-84e2790d49ec nodeName:}" failed. No retries permitted until 2026-09-08 02:21:54.645159601 +0000 UTC m=+10.176804458 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs") pod "network-metrics-daemon-mt4gl" (UID: "ec8f0aba-d0c2-4d45-b25d-84e2790d49ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 02:21:50.645791 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:50.645671 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 02:21:50.645791 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:50.645699 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 02:21:50.645791 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:50.645713 2560 projected.go:196] Error preparing data for projected volume kube-api-access-w2987 for pod openshift-network-diagnostics/network-check-target-c8fz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 02:21:50.645791 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:50.645765 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987 podName:fe2f6a1b-8c73-4c40-8525-d1329e889f46 nodeName:}" failed. No retries permitted until 2026-09-08 02:21:54.645750167 +0000 UTC m=+10.177395040 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-w2987" (UniqueName: "kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987") pod "network-check-target-c8fz9" (UID: "fe2f6a1b-8c73-4c40-8525-d1329e889f46") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 02:21:51.016620 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:51.016284 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:21:51.016620 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:51.016411 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmw7v" podUID="0af6038b-8bad-4d6a-9478-083edfbc974a" Sep 08 02:21:51.017507 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:51.017183 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:21:51.017507 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:51.017284 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c8fz9" podUID="fe2f6a1b-8c73-4c40-8525-d1329e889f46" Sep 08 02:21:51.017507 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:51.017347 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:21:51.017507 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:51.017436 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mt4gl" podUID="ec8f0aba-d0c2-4d45-b25d-84e2790d49ec" Sep 08 02:21:53.013652 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:53.013617 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:21:53.014105 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:53.013776 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mt4gl" podUID="ec8f0aba-d0c2-4d45-b25d-84e2790d49ec" Sep 08 02:21:53.014105 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:53.013962 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:21:53.014105 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:53.014036 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c8fz9" podUID="fe2f6a1b-8c73-4c40-8525-d1329e889f46" Sep 08 02:21:53.014105 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:53.014095 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:21:53.014349 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:53.014174 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmw7v" podUID="0af6038b-8bad-4d6a-9478-083edfbc974a" Sep 08 02:21:54.582371 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:54.582329 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret\") pod \"global-pull-secret-syncer-pmw7v\" (UID: \"0af6038b-8bad-4d6a-9478-083edfbc974a\") " pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:21:54.582839 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:54.582531 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 02:21:54.582839 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:54.582594 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret podName:0af6038b-8bad-4d6a-9478-083edfbc974a nodeName:}" failed. No retries permitted until 2026-09-08 02:22:02.582574898 +0000 UTC m=+18.114219744 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret") pod "global-pull-secret-syncer-pmw7v" (UID: "0af6038b-8bad-4d6a-9478-083edfbc974a") : object "kube-system"/"original-pull-secret" not registered Sep 08 02:21:54.682766 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:54.682730 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2987\" (UniqueName: \"kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987\") pod \"network-check-target-c8fz9\" (UID: \"fe2f6a1b-8c73-4c40-8525-d1329e889f46\") " pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:21:54.682947 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:54.682806 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs\") pod \"network-metrics-daemon-mt4gl\" (UID: \"ec8f0aba-d0c2-4d45-b25d-84e2790d49ec\") " pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:21:54.682947 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:54.682940 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 02:21:54.683061 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:54.683000 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs podName:ec8f0aba-d0c2-4d45-b25d-84e2790d49ec nodeName:}" failed. No retries permitted until 2026-09-08 02:22:02.682981693 +0000 UTC m=+18.214626555 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs") pod "network-metrics-daemon-mt4gl" (UID: "ec8f0aba-d0c2-4d45-b25d-84e2790d49ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 02:21:54.683761 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:54.683385 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 02:21:54.683761 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:54.683402 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 02:21:54.683761 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:54.683417 2560 projected.go:196] Error preparing data for projected volume kube-api-access-w2987 for pod openshift-network-diagnostics/network-check-target-c8fz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 02:21:54.683761 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:54.683455 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987 podName:fe2f6a1b-8c73-4c40-8525-d1329e889f46 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:02.683442889 +0000 UTC m=+18.215087735 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-w2987" (UniqueName: "kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987") pod "network-check-target-c8fz9" (UID: "fe2f6a1b-8c73-4c40-8525-d1329e889f46") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 02:21:55.015876 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:55.015567 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:21:55.016024 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:55.015923 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmw7v" podUID="0af6038b-8bad-4d6a-9478-083edfbc974a" Sep 08 02:21:55.016024 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:55.015761 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:21:55.016105 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:55.016029 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mt4gl" podUID="ec8f0aba-d0c2-4d45-b25d-84e2790d49ec" Sep 08 02:21:55.016105 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:55.015709 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:21:55.016200 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:55.016114 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c8fz9" podUID="fe2f6a1b-8c73-4c40-8525-d1329e889f46" Sep 08 02:21:57.015328 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:57.015203 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:21:57.015328 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:57.015228 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:21:57.015852 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:57.015333 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmw7v" podUID="0af6038b-8bad-4d6a-9478-083edfbc974a" Sep 08 02:21:57.015852 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:57.015379 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c8fz9" podUID="fe2f6a1b-8c73-4c40-8525-d1329e889f46" Sep 08 02:21:57.015852 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:57.015391 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:21:57.015852 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:57.015497 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mt4gl" podUID="ec8f0aba-d0c2-4d45-b25d-84e2790d49ec" Sep 08 02:21:59.011139 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:59.011105 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:21:59.011594 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:59.011111 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:21:59.011594 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:59.011235 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmw7v" podUID="0af6038b-8bad-4d6a-9478-083edfbc974a" Sep 08 02:21:59.011594 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:21:59.011111 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:21:59.011594 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:59.011298 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mt4gl" podUID="ec8f0aba-d0c2-4d45-b25d-84e2790d49ec" Sep 08 02:21:59.011594 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:21:59.011405 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c8fz9" podUID="fe2f6a1b-8c73-4c40-8525-d1329e889f46" Sep 08 02:22:01.011210 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:01.011177 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:22:01.011719 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:01.011178 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:22:01.011719 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:01.011295 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmw7v" podUID="0af6038b-8bad-4d6a-9478-083edfbc974a" Sep 08 02:22:01.011719 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:01.011439 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mt4gl" podUID="ec8f0aba-d0c2-4d45-b25d-84e2790d49ec" Sep 08 02:22:01.011719 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:01.011494 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:22:01.011719 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:01.011655 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c8fz9" podUID="fe2f6a1b-8c73-4c40-8525-d1329e889f46" Sep 08 02:22:02.641259 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:02.641221 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret\") pod \"global-pull-secret-syncer-pmw7v\" (UID: \"0af6038b-8bad-4d6a-9478-083edfbc974a\") " pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:22:02.641750 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:02.641376 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 02:22:02.641750 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:02.641428 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret podName:0af6038b-8bad-4d6a-9478-083edfbc974a nodeName:}" failed. No retries permitted until 2026-09-08 02:22:18.641413987 +0000 UTC m=+34.173058831 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret") pod "global-pull-secret-syncer-pmw7v" (UID: "0af6038b-8bad-4d6a-9478-083edfbc974a") : object "kube-system"/"original-pull-secret" not registered Sep 08 02:22:02.742417 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:02.742385 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2987\" (UniqueName: \"kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987\") pod \"network-check-target-c8fz9\" (UID: \"fe2f6a1b-8c73-4c40-8525-d1329e889f46\") " pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:22:02.742584 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:02.742437 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs\") pod \"network-metrics-daemon-mt4gl\" (UID: \"ec8f0aba-d0c2-4d45-b25d-84e2790d49ec\") " pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:22:02.742584 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:02.742539 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 02:22:02.742660 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:02.742591 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs podName:ec8f0aba-d0c2-4d45-b25d-84e2790d49ec nodeName:}" failed. No retries permitted until 2026-09-08 02:22:18.742577215 +0000 UTC m=+34.274222058 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs") pod "network-metrics-daemon-mt4gl" (UID: "ec8f0aba-d0c2-4d45-b25d-84e2790d49ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 02:22:02.742660 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:02.742592 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 02:22:02.742660 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:02.742612 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 02:22:02.742660 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:02.742624 2560 projected.go:196] Error preparing data for projected volume kube-api-access-w2987 for pod openshift-network-diagnostics/network-check-target-c8fz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 02:22:02.742660 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:02.742660 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987 podName:fe2f6a1b-8c73-4c40-8525-d1329e889f46 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:18.742650322 +0000 UTC m=+34.274295166 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-w2987" (UniqueName: "kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987") pod "network-check-target-c8fz9" (UID: "fe2f6a1b-8c73-4c40-8525-d1329e889f46") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 02:22:03.011583 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:03.011494 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:22:03.011732 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:03.011648 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mt4gl" podUID="ec8f0aba-d0c2-4d45-b25d-84e2790d49ec" Sep 08 02:22:03.011732 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:03.011494 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:22:03.011825 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:03.011754 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmw7v" podUID="0af6038b-8bad-4d6a-9478-083edfbc974a" Sep 08 02:22:03.011825 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:03.011494 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:22:03.011899 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:03.011886 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c8fz9" podUID="fe2f6a1b-8c73-4c40-8525-d1329e889f46" Sep 08 02:22:05.011767 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:05.011732 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:22:05.012260 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:05.011842 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c8fz9" podUID="fe2f6a1b-8c73-4c40-8525-d1329e889f46" Sep 08 02:22:05.012260 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:05.011857 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:22:05.012260 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:05.011881 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:22:05.012260 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:05.011987 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mt4gl" podUID="ec8f0aba-d0c2-4d45-b25d-84e2790d49ec" Sep 08 02:22:05.012260 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:05.012078 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmw7v" podUID="0af6038b-8bad-4d6a-9478-083edfbc974a" Sep 08 02:22:06.209934 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:06.209829 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x6s55" event={"ID":"76080513-3119-49e3-b19d-8c1a0d6f854c","Type":"ContainerStarted","Data":"bd6b16a740e1d24ca5db51a435360834b851ef8fe3e4aa910f34fbe5999fd7f2"} Sep 08 02:22:06.216109 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:06.216078 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gp9px" event={"ID":"550f906c-4554-42d1-9ef7-1922f2501656","Type":"ContainerStarted","Data":"9c39bcfe5bfc008c7a122430f2067f92bb90719a4597e94235d643fbfc57f4f0"} Sep 08 02:22:06.222549 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:06.222507 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8mcwm" event={"ID":"ab6018f8-3012-4ec8-959e-95479604e898","Type":"ContainerStarted","Data":"87d2898e8a86999179524bc6491b81dfba8d2af51de657f77f16cfb9b5de2a46"} Sep 08 02:22:06.230369 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:06.229560 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-x6s55" podStartSLOduration=3.459930476 podStartE2EDuration="21.22943413s" totalImagesPullingTime="17.769503654s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 02:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:22:06.229163009 +0000 UTC m=+21.760807891" watchObservedRunningTime="2026-09-08 02:22:06.22943413 +0000 UTC m=+21.761078999" Sep 08 02:22:06.231177 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:06.230829 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" podStartSLOduration=20.230813782 podStartE2EDuration="20.230813782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 02:21:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:21:50.188198005 +0000 UTC m=+5.719842875" watchObservedRunningTime="2026-09-08 02:22:06.230813782 +0000 UTC m=+21.762458649" Sep 08 02:22:06.247003 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:06.246941 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-66vxb" podStartSLOduration=3.483320899 podStartE2EDuration="21.24691968s" totalImagesPullingTime="17.763598781s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 02:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:22:06.246763177 +0000 UTC m=+21.778408045" watchObservedRunningTime="2026-09-08 02:22:06.24691968 +0000 UTC m=+21.778564546" Sep 08 02:22:06.263299 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:06.263102 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gp9px" podStartSLOduration=10.655998542 podStartE2EDuration="21.263085573s" totalImagesPullingTime="10.607087031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:22:06.262414541 +0000 UTC m=+21.794059406" watchObservedRunningTime="2026-09-08 02:22:06.263085573 +0000 UTC m=+21.794730492" Sep 08 02:22:06.301318 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:06.301065 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-252ld" podStartSLOduration=3.481435685 podStartE2EDuration="21.30104685s" totalImagesPullingTime="17.819611165s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 02:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:22:06.300816654 +0000 UTC m=+21.832461570" watchObservedRunningTime="2026-09-08 02:22:06.30104685 +0000 UTC m=+21.832691715" Sep 08 02:22:07.014119 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:07.014042 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:22:07.014119 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:07.014074 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:22:07.014119 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:07.014078 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:22:07.014348 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:07.014170 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmw7v" podUID="0af6038b-8bad-4d6a-9478-083edfbc974a" Sep 08 02:22:07.014348 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:07.014313 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c8fz9" podUID="fe2f6a1b-8c73-4c40-8525-d1329e889f46" Sep 08 02:22:07.014449 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:07.014410 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mt4gl" podUID="ec8f0aba-d0c2-4d45-b25d-84e2790d49ec" Sep 08 02:22:07.225536 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:07.225499 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" event={"ID":"6b9a058d-34e6-4e9e-9623-afbc7e0fa776","Type":"ContainerStarted","Data":"0051778745fae4c507e34a11ca1022ac5420667ba6ce50a513215cbdc429a050"} Sep 08 02:22:07.226908 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:07.226877 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-252ld" event={"ID":"196615ff-9773-44ba-b5a7-a56229dafb40","Type":"ContainerStarted","Data":"2c6d97bdd881de99e676cdbc3d7edddd8049cff55095c124ef177f15683e7267"} Sep 08 02:22:07.228690 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:07.228548 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vn7fg" event={"ID":"6fe0dc65-7890-405f-9c55-755957407695","Type":"ContainerStarted","Data":"35e9b8bdd64ad2aeafce761f466594e7011c2f712f3bbddbb512c640b04758e1"} Sep 08 02:22:07.232206 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:07.232182 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-66vxb" event={"ID":"185c152f-08c1-48bf-9d40-f61ee106ff12","Type":"ContainerStarted","Data":"92e2f89fe2be68c39942bd6d4160e7d655841900aa28c19a456448a39f44c2ed"} Sep 08 02:22:07.233585 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:07.233560 2560 generic.go:356] "Generic (PLEG): container finished" podID="ab6018f8-3012-4ec8-959e-95479604e898" containerID="87d2898e8a86999179524bc6491b81dfba8d2af51de657f77f16cfb9b5de2a46" exitCode=0 Sep 08 02:22:07.233688 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:07.233647 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8mcwm" event={"ID":"ab6018f8-3012-4ec8-959e-95479604e898","Type":"ContainerDied","Data":"87d2898e8a86999179524bc6491b81dfba8d2af51de657f77f16cfb9b5de2a46"} Sep 08 02:22:07.236510 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:07.236490 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" event={"ID":"05f8db49-040c-460f-b931-90988ff34ae7","Type":"ContainerStarted","Data":"62429a9dd307720702bbda525845f0dd6634be55fc14ad01a23995d6de05a056"} Sep 08 02:22:07.236598 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:07.236552 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" event={"ID":"05f8db49-040c-460f-b931-90988ff34ae7","Type":"ContainerStarted","Data":"f8612be9f44a7edd2b8aa8d4f2b59661f1bd94bd0cdc777b5d650a4f62bfb598"} Sep 08 02:22:07.236598 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:07.236583 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" event={"ID":"05f8db49-040c-460f-b931-90988ff34ae7","Type":"ContainerStarted","Data":"d9af74982c755a5c7dcbc987ec2165d140ff587030099ddc6a7b7d2d2f12659d"} Sep 08 02:22:07.236687 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:07.236611 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" event={"ID":"05f8db49-040c-460f-b931-90988ff34ae7","Type":"ContainerStarted","Data":"3b077a276346b32a85ae3d4586fa1fa6de1c4a7c2fa91bdc03960d09624849b6"} Sep 08 02:22:07.236687 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:07.236643 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" event={"ID":"05f8db49-040c-460f-b931-90988ff34ae7","Type":"ContainerStarted","Data":"bdad2b212a46e2b409e35801e154b441ffdf4c8452bd9ba0041f55742e40f3ab"} Sep 08 02:22:07.236687 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:07.236675 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" event={"ID":"05f8db49-040c-460f-b931-90988ff34ae7","Type":"ContainerStarted","Data":"8ed98e5215c15e4bcc9b29d2369623d2810f1ac3b31572987407777edbbce7af"} Sep 08 02:22:07.246140 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:07.246102 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vn7fg" podStartSLOduration=4.48117504 podStartE2EDuration="22.24609013s" totalImagesPullingTime="17.76491509s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 02:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:22:07.245875975 +0000 UTC m=+22.777520839" watchObservedRunningTime="2026-09-08 02:22:07.24609013 +0000 UTC m=+22.777734994" Sep 08 02:22:07.396426 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:07.396220 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 08 02:22:07.826362 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:07.826317 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gp9px" Sep 08 02:22:07.826990 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:07.826970 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gp9px" Sep 08 02:22:07.944429 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:07.944321 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-08T02:22:07.396421675Z","UUID":"d7907003-3d7d-44a8-8b6e-0c087723560f","Handler":null,"Name":"","Endpoint":""} Sep 08 02:22:07.946735 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:07.946574 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 08 02:22:07.946735 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:07.946612 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 08 02:22:08.240481 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:08.240381 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" event={"ID":"6b9a058d-34e6-4e9e-9623-afbc7e0fa776","Type":"ContainerStarted","Data":"f15f76f0b811754548f29d8979749994870aa96dd7b55ef39b3b971beb1eb870"} Sep 08 02:22:08.242096 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:08.242067 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8w8k7" event={"ID":"bdf9207b-a3c4-4579-abdc-de9a2465f484","Type":"ContainerStarted","Data":"86777f67863375a5c378fc4fb857ad9061e7c7021ea378458ef66847b0abd43b"} Sep 08 02:22:08.242645 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:08.242621 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gp9px" Sep 08 02:22:08.243088 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:08.243067 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gp9px" Sep 08 02:22:08.260286 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:08.260232 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8w8k7" podStartSLOduration=5.505926265 podStartE2EDuration="23.260215498s" totalImagesPullingTime="17.754289233s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 02:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:22:08.259654852 +0000 UTC m=+23.791299716" watchObservedRunningTime="2026-09-08 02:22:08.260215498 +0000 UTC m=+23.791860363" Sep 08 02:22:09.012014 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:09.011931 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:22:09.012163 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:09.011930 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:22:09.012163 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:09.012072 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c8fz9" podUID="fe2f6a1b-8c73-4c40-8525-d1329e889f46" Sep 08 02:22:09.012270 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:09.012185 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mt4gl" podUID="ec8f0aba-d0c2-4d45-b25d-84e2790d49ec" Sep 08 02:22:09.012270 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:09.011937 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:22:09.012363 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:09.012317 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmw7v" podUID="0af6038b-8bad-4d6a-9478-083edfbc974a" Sep 08 02:22:09.246628 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:09.246587 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" event={"ID":"6b9a058d-34e6-4e9e-9623-afbc7e0fa776","Type":"ContainerStarted","Data":"674e586fc1b3ff4361437868cd3f52da63e85cceb8062e198f203f33f977b6a1"} Sep 08 02:22:09.250558 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:09.250526 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" event={"ID":"05f8db49-040c-460f-b931-90988ff34ae7","Type":"ContainerStarted","Data":"d7a5851c633be26d8860d340f2494e3571b902d0d6dfd0712204219baa483a66"} Sep 08 02:22:09.265828 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:09.265755 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lzpmq" podStartSLOduration=4.536564122 podStartE2EDuration="24.26574147s" totalImagesPullingTime="19.729177348s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 02:21:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:22:09.265373043 +0000 UTC m=+24.797017909" watchObservedRunningTime="2026-09-08 02:22:09.26574147 +0000 UTC m=+24.797386349" Sep 08 02:22:11.011117 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:11.011084 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:22:11.011651 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:11.011098 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:22:11.011651 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:11.011204 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c8fz9" podUID="fe2f6a1b-8c73-4c40-8525-d1329e889f46" Sep 08 02:22:11.011651 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:11.011296 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mt4gl" podUID="ec8f0aba-d0c2-4d45-b25d-84e2790d49ec" Sep 08 02:22:11.011651 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:11.011099 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:22:11.011651 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:11.011394 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmw7v" podUID="0af6038b-8bad-4d6a-9478-083edfbc974a" Sep 08 02:22:12.259029 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:12.258801 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8mcwm" event={"ID":"ab6018f8-3012-4ec8-959e-95479604e898","Type":"ContainerStarted","Data":"cc32c63efb2e98a8ad37210bcd400eaad90b92103ac9f54914368a73ca208564"} Sep 08 02:22:12.262686 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:12.262656 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" event={"ID":"05f8db49-040c-460f-b931-90988ff34ae7","Type":"ContainerStarted","Data":"56a225e923cab558f74773fb0ec47cd3444f5e2cadd71ba053c317bd8637d488"} Sep 08 02:22:12.262918 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:12.262899 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:22:12.277243 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:12.277212 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:22:12.315005 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:12.314957 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" podStartSLOduration=9.500940809 podStartE2EDuration="27.314943905s" totalImagesPullingTime="17.814003096s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 02:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:22:12.313231449 +0000 UTC m=+27.844876314" watchObservedRunningTime="2026-09-08 02:22:12.314943905 +0000 UTC m=+27.846588770" Sep 08 02:22:13.011126 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:13.011098 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:22:13.011391 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:13.011098 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:22:13.011391 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:13.011192 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c8fz9" podUID="fe2f6a1b-8c73-4c40-8525-d1329e889f46" Sep 08 02:22:13.011391 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:13.011224 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:22:13.011391 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:13.011305 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmw7v" podUID="0af6038b-8bad-4d6a-9478-083edfbc974a" Sep 08 02:22:13.011391 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:13.011369 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mt4gl" podUID="ec8f0aba-d0c2-4d45-b25d-84e2790d49ec" Sep 08 02:22:13.267205 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:13.267117 2560 generic.go:356] "Generic (PLEG): container finished" podID="ab6018f8-3012-4ec8-959e-95479604e898" containerID="cc32c63efb2e98a8ad37210bcd400eaad90b92103ac9f54914368a73ca208564" exitCode=0 Sep 08 02:22:13.267205 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:13.267171 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8mcwm" event={"ID":"ab6018f8-3012-4ec8-959e-95479604e898","Type":"ContainerDied","Data":"cc32c63efb2e98a8ad37210bcd400eaad90b92103ac9f54914368a73ca208564"} Sep 08 02:22:13.267937 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:13.267689 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:22:13.267937 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:13.267752 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:22:13.283184 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:13.283157 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:22:13.874208 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:13.873947 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pmw7v"] Sep 08 02:22:13.874360 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:13.874287 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:22:13.874402 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:13.874384 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmw7v" podUID="0af6038b-8bad-4d6a-9478-083edfbc974a" Sep 08 02:22:13.876975 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:13.876951 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mt4gl"] Sep 08 02:22:13.877110 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:13.877044 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:22:13.877171 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:13.877126 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mt4gl" podUID="ec8f0aba-d0c2-4d45-b25d-84e2790d49ec" Sep 08 02:22:13.877544 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:13.877521 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c8fz9"] Sep 08 02:22:13.877664 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:13.877584 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:22:13.877664 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:13.877652 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c8fz9" podUID="fe2f6a1b-8c73-4c40-8525-d1329e889f46" Sep 08 02:22:15.011833 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:15.011795 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:22:15.012413 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:15.011902 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c8fz9" podUID="fe2f6a1b-8c73-4c40-8525-d1329e889f46" Sep 08 02:22:15.012413 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:15.011982 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:22:15.012413 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:15.012079 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mt4gl" podUID="ec8f0aba-d0c2-4d45-b25d-84e2790d49ec" Sep 08 02:22:15.012413 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:15.012130 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:22:15.012413 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:15.012199 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmw7v" podUID="0af6038b-8bad-4d6a-9478-083edfbc974a" Sep 08 02:22:15.274028 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:15.273950 2560 generic.go:356] "Generic (PLEG): container finished" podID="ab6018f8-3012-4ec8-959e-95479604e898" containerID="741b9d6d6190cba4054e5c0d133ddef05dddfae97fce8b2c883eb83554ad2b41" exitCode=0 Sep 08 02:22:15.274028 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:15.273992 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8mcwm" event={"ID":"ab6018f8-3012-4ec8-959e-95479604e898","Type":"ContainerDied","Data":"741b9d6d6190cba4054e5c0d133ddef05dddfae97fce8b2c883eb83554ad2b41"} Sep 08 02:22:16.278963 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:16.278866 2560 generic.go:356] "Generic (PLEG): container finished" podID="ab6018f8-3012-4ec8-959e-95479604e898" containerID="09983dc06b766fb5bd648c4eebbb2ebcb4472243aebeeab4fd150612a8ec9519" exitCode=0 Sep 08 02:22:16.278963 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:16.278934 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8mcwm" event={"ID":"ab6018f8-3012-4ec8-959e-95479604e898","Type":"ContainerDied","Data":"09983dc06b766fb5bd648c4eebbb2ebcb4472243aebeeab4fd150612a8ec9519"} Sep 08 02:22:17.011680 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:17.011581 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:22:17.011680 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:17.011581 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:22:17.011896 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:17.011705 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pmw7v" podUID="0af6038b-8bad-4d6a-9478-083edfbc974a" Sep 08 02:22:17.011896 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:17.011592 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:22:17.011896 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:17.011765 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c8fz9" podUID="fe2f6a1b-8c73-4c40-8525-d1329e889f46" Sep 08 02:22:17.011896 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:17.011872 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mt4gl" podUID="ec8f0aba-d0c2-4d45-b25d-84e2790d49ec" Sep 08 02:22:18.655271 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.655238 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret\") pod \"global-pull-secret-syncer-pmw7v\" (UID: \"0af6038b-8bad-4d6a-9478-083edfbc974a\") " pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:22:18.655807 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:18.655367 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 02:22:18.655807 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:18.655430 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret podName:0af6038b-8bad-4d6a-9478-083edfbc974a nodeName:}" failed. No retries permitted until 2026-09-08 02:22:50.65541423 +0000 UTC m=+66.187059075 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret") pod "global-pull-secret-syncer-pmw7v" (UID: "0af6038b-8bad-4d6a-9478-083edfbc974a") : object "kube-system"/"original-pull-secret" not registered Sep 08 02:22:18.694323 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.694294 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeReady" Sep 08 02:22:18.694485 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.694408 2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 08 02:22:18.756418 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.756382 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2987\" (UniqueName: \"kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987\") pod \"network-check-target-c8fz9\" (UID: \"fe2f6a1b-8c73-4c40-8525-d1329e889f46\") " pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:22:18.756616 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.756450 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs\") pod \"network-metrics-daemon-mt4gl\" (UID: \"ec8f0aba-d0c2-4d45-b25d-84e2790d49ec\") " pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:22:18.756616 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:18.756578 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 02:22:18.756616 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:18.756593 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 02:22:18.756766 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:18.756624 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 02:22:18.756766 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:18.756639 2560 projected.go:196] Error preparing data for projected volume kube-api-access-w2987 for pod openshift-network-diagnostics/network-check-target-c8fz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 02:22:18.756766 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:18.756646 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs podName:ec8f0aba-d0c2-4d45-b25d-84e2790d49ec nodeName:}" failed. No retries permitted until 2026-09-08 02:22:50.75662822 +0000 UTC m=+66.288273072 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs") pod "network-metrics-daemon-mt4gl" (UID: "ec8f0aba-d0c2-4d45-b25d-84e2790d49ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 02:22:18.756766 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:18.756714 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987 podName:fe2f6a1b-8c73-4c40-8525-d1329e889f46 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:50.756697528 +0000 UTC m=+66.288342377 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-w2987" (UniqueName: "kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987") pod "network-check-target-c8fz9" (UID: "fe2f6a1b-8c73-4c40-8525-d1329e889f46") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 02:22:18.760581 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.760553 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qhkj2"] Sep 08 02:22:18.763392 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.763370 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-p6qlw"] Sep 08 02:22:18.763546 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.763525 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qhkj2" Sep 08 02:22:18.766224 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.766201 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p6qlw" Sep 08 02:22:18.767320 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.767300 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 08 02:22:18.767320 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.767308 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-56wgd\"" Sep 08 02:22:18.767658 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.767637 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 08 02:22:18.768981 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.768961 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 08 02:22:18.769126 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.769021 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xsgpb\"" Sep 08 02:22:18.769126 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.769049 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 08 02:22:18.769310 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.769275 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 08 02:22:18.775926 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.775903 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p6qlw"] Sep 08 02:22:18.778568 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.778547 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qhkj2"] Sep 08 02:22:18.857021 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.856936 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aeddc1eb-1d10-4d91-a627-6d0cae0ad53b-cert\") pod \"ingress-canary-p6qlw\" (UID: \"aeddc1eb-1d10-4d91-a627-6d0cae0ad53b\") " pod="openshift-ingress-canary/ingress-canary-p6qlw" Sep 08 02:22:18.857021 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.856986 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/382809a0-5fcd-4635-8ab5-f7e83672b1a5-config-volume\") pod \"dns-default-qhkj2\" (UID: \"382809a0-5fcd-4635-8ab5-f7e83672b1a5\") " pod="openshift-dns/dns-default-qhkj2" Sep 08 02:22:18.857230 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.857026 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls\") pod \"dns-default-qhkj2\" (UID: \"382809a0-5fcd-4635-8ab5-f7e83672b1a5\") " pod="openshift-dns/dns-default-qhkj2" Sep 08 02:22:18.857230 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.857046 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/382809a0-5fcd-4635-8ab5-f7e83672b1a5-tmp-dir\") pod \"dns-default-qhkj2\" (UID: \"382809a0-5fcd-4635-8ab5-f7e83672b1a5\") " pod="openshift-dns/dns-default-qhkj2" Sep 08 02:22:18.857230 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.857090 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92nns\" (UniqueName: \"kubernetes.io/projected/382809a0-5fcd-4635-8ab5-f7e83672b1a5-kube-api-access-92nns\") pod \"dns-default-qhkj2\" (UID: \"382809a0-5fcd-4635-8ab5-f7e83672b1a5\") " pod="openshift-dns/dns-default-qhkj2" Sep 08 02:22:18.857230 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.857119 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft7xx\" (UniqueName: \"kubernetes.io/projected/aeddc1eb-1d10-4d91-a627-6d0cae0ad53b-kube-api-access-ft7xx\") pod \"ingress-canary-p6qlw\" (UID: \"aeddc1eb-1d10-4d91-a627-6d0cae0ad53b\") " pod="openshift-ingress-canary/ingress-canary-p6qlw" Sep 08 02:22:18.957435 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.957398 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft7xx\" (UniqueName: \"kubernetes.io/projected/aeddc1eb-1d10-4d91-a627-6d0cae0ad53b-kube-api-access-ft7xx\") pod \"ingress-canary-p6qlw\" (UID: \"aeddc1eb-1d10-4d91-a627-6d0cae0ad53b\") " pod="openshift-ingress-canary/ingress-canary-p6qlw" Sep 08 02:22:18.957599 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.957452 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aeddc1eb-1d10-4d91-a627-6d0cae0ad53b-cert\") pod \"ingress-canary-p6qlw\" (UID: \"aeddc1eb-1d10-4d91-a627-6d0cae0ad53b\") " pod="openshift-ingress-canary/ingress-canary-p6qlw" Sep 08 02:22:18.957682 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.957668 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/382809a0-5fcd-4635-8ab5-f7e83672b1a5-config-volume\") pod \"dns-default-qhkj2\" (UID: \"382809a0-5fcd-4635-8ab5-f7e83672b1a5\") " pod="openshift-dns/dns-default-qhkj2" Sep 08 02:22:18.957720 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.957693 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls\") pod \"dns-default-qhkj2\" (UID: \"382809a0-5fcd-4635-8ab5-f7e83672b1a5\") " pod="openshift-dns/dns-default-qhkj2" Sep 08 02:22:18.957720 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.957707 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/382809a0-5fcd-4635-8ab5-f7e83672b1a5-tmp-dir\") pod \"dns-default-qhkj2\" (UID: \"382809a0-5fcd-4635-8ab5-f7e83672b1a5\") " pod="openshift-dns/dns-default-qhkj2" Sep 08 02:22:18.957797 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.957756 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92nns\" (UniqueName: \"kubernetes.io/projected/382809a0-5fcd-4635-8ab5-f7e83672b1a5-kube-api-access-92nns\") pod \"dns-default-qhkj2\" (UID: \"382809a0-5fcd-4635-8ab5-f7e83672b1a5\") " pod="openshift-dns/dns-default-qhkj2" Sep 08 02:22:18.957846 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:18.957819 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 02:22:18.957896 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:18.957885 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls podName:382809a0-5fcd-4635-8ab5-f7e83672b1a5 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:19.457870743 +0000 UTC m=+34.989515590 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls") pod "dns-default-qhkj2" (UID: "382809a0-5fcd-4635-8ab5-f7e83672b1a5") : secret "dns-default-metrics-tls" not found Sep 08 02:22:18.958107 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.958086 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/382809a0-5fcd-4635-8ab5-f7e83672b1a5-tmp-dir\") pod \"dns-default-qhkj2\" (UID: \"382809a0-5fcd-4635-8ab5-f7e83672b1a5\") " pod="openshift-dns/dns-default-qhkj2" Sep 08 02:22:18.958359 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.958342 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/382809a0-5fcd-4635-8ab5-f7e83672b1a5-config-volume\") pod \"dns-default-qhkj2\" (UID: \"382809a0-5fcd-4635-8ab5-f7e83672b1a5\") " pod="openshift-dns/dns-default-qhkj2" Sep 08 02:22:18.961895 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.961746 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aeddc1eb-1d10-4d91-a627-6d0cae0ad53b-cert\") pod \"ingress-canary-p6qlw\" (UID: \"aeddc1eb-1d10-4d91-a627-6d0cae0ad53b\") " pod="openshift-ingress-canary/ingress-canary-p6qlw" Sep 08 02:22:18.966937 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.966914 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92nns\" (UniqueName: \"kubernetes.io/projected/382809a0-5fcd-4635-8ab5-f7e83672b1a5-kube-api-access-92nns\") pod \"dns-default-qhkj2\" (UID: \"382809a0-5fcd-4635-8ab5-f7e83672b1a5\") " pod="openshift-dns/dns-default-qhkj2" Sep 08 02:22:18.967304 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:18.967288 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft7xx\" (UniqueName: \"kubernetes.io/projected/aeddc1eb-1d10-4d91-a627-6d0cae0ad53b-kube-api-access-ft7xx\") pod \"ingress-canary-p6qlw\" (UID: \"aeddc1eb-1d10-4d91-a627-6d0cae0ad53b\") " pod="openshift-ingress-canary/ingress-canary-p6qlw" Sep 08 02:22:19.011459 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:19.011426 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:22:19.011459 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:19.011434 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:22:19.011679 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:19.011574 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:22:19.014855 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:19.014832 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 08 02:22:19.015034 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:19.014877 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 08 02:22:19.015034 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:19.014835 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 08 02:22:19.015139 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:19.015082 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m9nsl\"" Sep 08 02:22:19.015139 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:19.015103 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 08 02:22:19.015295 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:19.015276 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-42xzn\"" Sep 08 02:22:19.083863 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:19.083801 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p6qlw" Sep 08 02:22:19.374577 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:19.374547 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p6qlw"] Sep 08 02:22:19.377919 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:22:19.377897 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeddc1eb_1d10_4d91_a627_6d0cae0ad53b.slice/crio-1d6db9fa4608bbdfa73dce7f54d38879daf650684efd129f4f5004ec93f1a21f WatchSource:0}: Error finding container 1d6db9fa4608bbdfa73dce7f54d38879daf650684efd129f4f5004ec93f1a21f: Status 404 returned error can't find the container with id 1d6db9fa4608bbdfa73dce7f54d38879daf650684efd129f4f5004ec93f1a21f Sep 08 02:22:19.461798 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:19.461720 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls\") pod \"dns-default-qhkj2\" (UID: \"382809a0-5fcd-4635-8ab5-f7e83672b1a5\") " pod="openshift-dns/dns-default-qhkj2" Sep 08 02:22:19.461947 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:19.461847 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 02:22:19.461947 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:19.461904 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls podName:382809a0-5fcd-4635-8ab5-f7e83672b1a5 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:20.461887558 +0000 UTC m=+35.993532405 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls") pod "dns-default-qhkj2" (UID: "382809a0-5fcd-4635-8ab5-f7e83672b1a5") : secret "dns-default-metrics-tls" not found Sep 08 02:22:20.289882 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:20.289843 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p6qlw" event={"ID":"aeddc1eb-1d10-4d91-a627-6d0cae0ad53b","Type":"ContainerStarted","Data":"1d6db9fa4608bbdfa73dce7f54d38879daf650684efd129f4f5004ec93f1a21f"} Sep 08 02:22:20.469472 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:20.469422 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls\") pod \"dns-default-qhkj2\" (UID: \"382809a0-5fcd-4635-8ab5-f7e83672b1a5\") " pod="openshift-dns/dns-default-qhkj2" Sep 08 02:22:20.469645 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:20.469593 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 02:22:20.469695 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:20.469662 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls podName:382809a0-5fcd-4635-8ab5-f7e83672b1a5 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:22.469643386 +0000 UTC m=+38.001288251 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls") pod "dns-default-qhkj2" (UID: "382809a0-5fcd-4635-8ab5-f7e83672b1a5") : secret "dns-default-metrics-tls" not found Sep 08 02:22:22.298085 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:22.298045 2560 generic.go:356] "Generic (PLEG): container finished" podID="ab6018f8-3012-4ec8-959e-95479604e898" containerID="cb9d3350ae3949eb35bad0e232b265163a98233117143b2c2d3493d76fcd1392" exitCode=0 Sep 08 02:22:22.298680 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:22.298110 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8mcwm" event={"ID":"ab6018f8-3012-4ec8-959e-95479604e898","Type":"ContainerDied","Data":"cb9d3350ae3949eb35bad0e232b265163a98233117143b2c2d3493d76fcd1392"} Sep 08 02:22:22.484249 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:22.484224 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls\") pod \"dns-default-qhkj2\" (UID: \"382809a0-5fcd-4635-8ab5-f7e83672b1a5\") " pod="openshift-dns/dns-default-qhkj2" Sep 08 02:22:22.484575 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:22.484554 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 02:22:22.484657 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:22.484630 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls podName:382809a0-5fcd-4635-8ab5-f7e83672b1a5 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:26.484606349 +0000 UTC m=+42.016251194 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls") pod "dns-default-qhkj2" (UID: "382809a0-5fcd-4635-8ab5-f7e83672b1a5") : secret "dns-default-metrics-tls" not found Sep 08 02:22:23.304389 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:23.304308 2560 generic.go:356] "Generic (PLEG): container finished" podID="ab6018f8-3012-4ec8-959e-95479604e898" containerID="b759efec41cfe7d2b8880f1396f6a1f8ed9577e83bdcdc45e6c03ffb0927685b" exitCode=0 Sep 08 02:22:23.304389 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:23.304369 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8mcwm" event={"ID":"ab6018f8-3012-4ec8-959e-95479604e898","Type":"ContainerDied","Data":"b759efec41cfe7d2b8880f1396f6a1f8ed9577e83bdcdc45e6c03ffb0927685b"} Sep 08 02:22:24.309896 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:24.309809 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8mcwm" event={"ID":"ab6018f8-3012-4ec8-959e-95479604e898","Type":"ContainerStarted","Data":"d0a53a9fb6825398e0cfd422980ad0a359e06936c231f80dd77d6670714492a7"} Sep 08 02:22:24.311272 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:24.311249 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p6qlw" event={"ID":"aeddc1eb-1d10-4d91-a627-6d0cae0ad53b","Type":"ContainerStarted","Data":"a28de818c8672f81c58357511c61421348bcdc3966ffba36385012bcf75d8de3"} Sep 08 02:22:24.336853 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:24.336810 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8mcwm" podStartSLOduration=9.672405483 podStartE2EDuration="39.336795721s" totalImagesPullingTime="29.664390238s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 02:21:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:22:24.334836512 +0000 UTC m=+39.866481378" watchObservedRunningTime="2026-09-08 02:22:24.336795721 +0000 UTC m=+39.868440587" Sep 08 02:22:24.351192 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:24.351154 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-p6qlw" podStartSLOduration=1.7983355159999999 podStartE2EDuration="6.351142878s" totalImagesPullingTime="4.552807362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:22:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:22:24.350300069 +0000 UTC m=+39.881944935" watchObservedRunningTime="2026-09-08 02:22:24.351142878 +0000 UTC m=+39.882787733" Sep 08 02:22:26.513625 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:26.513582 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls\") pod \"dns-default-qhkj2\" (UID: \"382809a0-5fcd-4635-8ab5-f7e83672b1a5\") " pod="openshift-dns/dns-default-qhkj2" Sep 08 02:22:26.513975 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:26.513726 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 02:22:26.513975 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:26.513785 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls podName:382809a0-5fcd-4635-8ab5-f7e83672b1a5 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:34.513769056 +0000 UTC m=+50.045413912 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls") pod "dns-default-qhkj2" (UID: "382809a0-5fcd-4635-8ab5-f7e83672b1a5") : secret "dns-default-metrics-tls" not found Sep 08 02:22:34.563820 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:34.563784 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls\") pod \"dns-default-qhkj2\" (UID: \"382809a0-5fcd-4635-8ab5-f7e83672b1a5\") " pod="openshift-dns/dns-default-qhkj2" Sep 08 02:22:34.564327 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:34.563940 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 02:22:34.564327 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:34.564022 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls podName:382809a0-5fcd-4635-8ab5-f7e83672b1a5 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:50.563999075 +0000 UTC m=+66.095643917 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls") pod "dns-default-qhkj2" (UID: "382809a0-5fcd-4635-8ab5-f7e83672b1a5") : secret "dns-default-metrics-tls" not found Sep 08 02:22:39.675842 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.675804 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-2hccm"] Sep 08 02:22:39.677729 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.677706 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" Sep 08 02:22:39.680957 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.680932 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-l7xv8\"" Sep 08 02:22:39.680957 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.680950 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 08 02:22:39.682057 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.682037 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 08 02:22:39.682135 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.682103 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 08 02:22:39.682187 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.682103 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 08 02:22:39.687810 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.687792 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 08 02:22:39.689898 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.689877 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-2hccm"] Sep 08 02:22:39.697475 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.697444 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/312a0520-e0c1-45b6-a9ac-e9f0b54df733-snapshots\") pod \"insights-operator-76bcd6ccf5-2hccm\" (UID: \"312a0520-e0c1-45b6-a9ac-e9f0b54df733\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" Sep 08 02:22:39.697553 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.697526 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/312a0520-e0c1-45b6-a9ac-e9f0b54df733-tmp\") pod \"insights-operator-76bcd6ccf5-2hccm\" (UID: \"312a0520-e0c1-45b6-a9ac-e9f0b54df733\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" Sep 08 02:22:39.697592 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.697561 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4fbx\" (UniqueName: \"kubernetes.io/projected/312a0520-e0c1-45b6-a9ac-e9f0b54df733-kube-api-access-m4fbx\") pod \"insights-operator-76bcd6ccf5-2hccm\" (UID: \"312a0520-e0c1-45b6-a9ac-e9f0b54df733\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" Sep 08 02:22:39.697625 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.697593 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/312a0520-e0c1-45b6-a9ac-e9f0b54df733-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-2hccm\" (UID: \"312a0520-e0c1-45b6-a9ac-e9f0b54df733\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" Sep 08 02:22:39.697662 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.697640 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/312a0520-e0c1-45b6-a9ac-e9f0b54df733-serving-cert\") pod \"insights-operator-76bcd6ccf5-2hccm\" (UID: \"312a0520-e0c1-45b6-a9ac-e9f0b54df733\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" Sep 08 02:22:39.697694 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.697686 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/312a0520-e0c1-45b6-a9ac-e9f0b54df733-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-2hccm\" (UID: \"312a0520-e0c1-45b6-a9ac-e9f0b54df733\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" Sep 08 02:22:39.774942 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.774911 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-t257k"] Sep 08 02:22:39.776565 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.776542 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-t257k" Sep 08 02:22:39.779230 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.779188 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 08 02:22:39.779375 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.779328 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 08 02:22:39.779450 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.779423 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 08 02:22:39.779582 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.779564 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-r8hht\"" Sep 08 02:22:39.782780 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.782758 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-drqqr"] Sep 08 02:22:39.784187 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.784171 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-drqqr" Sep 08 02:22:39.787253 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.787231 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 08 02:22:39.787338 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.787304 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 08 02:22:39.787599 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.787585 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 08 02:22:39.787663 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.787590 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-j9qfk\"" Sep 08 02:22:39.787784 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.787772 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 08 02:22:39.791075 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.791054 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-t257k"] Sep 08 02:22:39.798080 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.798054 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/312a0520-e0c1-45b6-a9ac-e9f0b54df733-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-2hccm\" (UID: \"312a0520-e0c1-45b6-a9ac-e9f0b54df733\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" Sep 08 02:22:39.798203 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.798099 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/61eceefd-089d-4d1f-9088-1793fd4fe32d-config\") pod \"service-ca-operator-859f94c4b-drqqr\" (UID: \"61eceefd-089d-4d1f-9088-1793fd4fe32d\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-drqqr" Sep 08 02:22:39.798203 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.798146 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ade53f57-ad06-4ce6-aacf-8bcd0771e61e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-t257k\" (UID: \"ade53f57-ad06-4ce6-aacf-8bcd0771e61e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-t257k" Sep 08 02:22:39.798203 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.798174 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2fml\" (UniqueName: \"kubernetes.io/projected/61eceefd-089d-4d1f-9088-1793fd4fe32d-kube-api-access-s2fml\") pod \"service-ca-operator-859f94c4b-drqqr\" (UID: \"61eceefd-089d-4d1f-9088-1793fd4fe32d\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-drqqr" Sep 08 02:22:39.798560 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.798223 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/312a0520-e0c1-45b6-a9ac-e9f0b54df733-serving-cert\") pod \"insights-operator-76bcd6ccf5-2hccm\" (UID: \"312a0520-e0c1-45b6-a9ac-e9f0b54df733\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" Sep 08 02:22:39.798560 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.798254 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njc5l\" (UniqueName: \"kubernetes.io/projected/ade53f57-ad06-4ce6-aacf-8bcd0771e61e-kube-api-access-njc5l\") pod \"cluster-samples-operator-7cdcfcf76c-t257k\" (UID: \"ade53f57-ad06-4ce6-aacf-8bcd0771e61e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-t257k" Sep 08 02:22:39.798560 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.798283 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/312a0520-e0c1-45b6-a9ac-e9f0b54df733-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-2hccm\" (UID: \"312a0520-e0c1-45b6-a9ac-e9f0b54df733\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" Sep 08 02:22:39.798560 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.798311 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/312a0520-e0c1-45b6-a9ac-e9f0b54df733-snapshots\") pod \"insights-operator-76bcd6ccf5-2hccm\" (UID: \"312a0520-e0c1-45b6-a9ac-e9f0b54df733\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" Sep 08 02:22:39.798560 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.798353 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/312a0520-e0c1-45b6-a9ac-e9f0b54df733-tmp\") pod \"insights-operator-76bcd6ccf5-2hccm\" (UID: \"312a0520-e0c1-45b6-a9ac-e9f0b54df733\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" Sep 08 02:22:39.798560 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.798388 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61eceefd-089d-4d1f-9088-1793fd4fe32d-serving-cert\") pod \"service-ca-operator-859f94c4b-drqqr\" (UID: \"61eceefd-089d-4d1f-9088-1793fd4fe32d\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-drqqr" Sep 08 02:22:39.798560 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.798412 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4fbx\" (UniqueName: \"kubernetes.io/projected/312a0520-e0c1-45b6-a9ac-e9f0b54df733-kube-api-access-m4fbx\") pod \"insights-operator-76bcd6ccf5-2hccm\" (UID: \"312a0520-e0c1-45b6-a9ac-e9f0b54df733\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" Sep 08 02:22:39.798911 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.798808 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/312a0520-e0c1-45b6-a9ac-e9f0b54df733-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-2hccm\" (UID: \"312a0520-e0c1-45b6-a9ac-e9f0b54df733\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" Sep 08 02:22:39.799079 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.799060 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-drqqr"] Sep 08 02:22:39.799205 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.799134 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/312a0520-e0c1-45b6-a9ac-e9f0b54df733-tmp\") pod \"insights-operator-76bcd6ccf5-2hccm\" (UID: \"312a0520-e0c1-45b6-a9ac-e9f0b54df733\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" Sep 08 02:22:39.799479 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.799435 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/312a0520-e0c1-45b6-a9ac-e9f0b54df733-snapshots\") pod \"insights-operator-76bcd6ccf5-2hccm\" (UID: \"312a0520-e0c1-45b6-a9ac-e9f0b54df733\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" Sep 08 02:22:39.799673 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.799651 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/312a0520-e0c1-45b6-a9ac-e9f0b54df733-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-2hccm\" (UID: \"312a0520-e0c1-45b6-a9ac-e9f0b54df733\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" Sep 08 02:22:39.801995 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.801969 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/312a0520-e0c1-45b6-a9ac-e9f0b54df733-serving-cert\") pod \"insights-operator-76bcd6ccf5-2hccm\" (UID: \"312a0520-e0c1-45b6-a9ac-e9f0b54df733\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" Sep 08 02:22:39.815993 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.815968 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4fbx\" (UniqueName: \"kubernetes.io/projected/312a0520-e0c1-45b6-a9ac-e9f0b54df733-kube-api-access-m4fbx\") pod \"insights-operator-76bcd6ccf5-2hccm\" (UID: \"312a0520-e0c1-45b6-a9ac-e9f0b54df733\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" Sep 08 02:22:39.884170 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.884143 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5d6c5474d8-9wrc6"] Sep 08 02:22:39.886280 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.886265 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:39.889563 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.889542 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 08 02:22:39.889681 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.889659 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 08 02:22:39.889825 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.889808 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 08 02:22:39.889895 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.889832 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-hqj47\"" Sep 08 02:22:39.889895 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.889874 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 08 02:22:39.890077 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.890061 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 08 02:22:39.890136 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.890119 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 08 02:22:39.899455 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.899438 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/61eceefd-089d-4d1f-9088-1793fd4fe32d-config\") pod \"service-ca-operator-859f94c4b-drqqr\" (UID: \"61eceefd-089d-4d1f-9088-1793fd4fe32d\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-drqqr" Sep 08 02:22:39.899557 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.899479 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ade53f57-ad06-4ce6-aacf-8bcd0771e61e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-t257k\" (UID: \"ade53f57-ad06-4ce6-aacf-8bcd0771e61e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-t257k" Sep 08 02:22:39.899557 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.899498 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2fml\" (UniqueName: \"kubernetes.io/projected/61eceefd-089d-4d1f-9088-1793fd4fe32d-kube-api-access-s2fml\") pod \"service-ca-operator-859f94c4b-drqqr\" (UID: \"61eceefd-089d-4d1f-9088-1793fd4fe32d\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-drqqr" Sep 08 02:22:39.899557 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.899518 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-default-certificate\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:39.899557 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.899551 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njc5l\" (UniqueName: \"kubernetes.io/projected/ade53f57-ad06-4ce6-aacf-8bcd0771e61e-kube-api-access-njc5l\") pod \"cluster-samples-operator-7cdcfcf76c-t257k\" (UID: \"ade53f57-ad06-4ce6-aacf-8bcd0771e61e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-t257k" Sep 08 02:22:39.899692 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.899573 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55e533a6-3267-4b11-966b-677e70829961-service-ca-bundle\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:39.899692 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.899590 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-metrics-certs\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:39.899692 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.899642 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61eceefd-089d-4d1f-9088-1793fd4fe32d-serving-cert\") pod \"service-ca-operator-859f94c4b-drqqr\" (UID: \"61eceefd-089d-4d1f-9088-1793fd4fe32d\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-drqqr" Sep 08 02:22:39.899692 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:39.899678 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 08 02:22:39.899692 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.899683 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-stats-auth\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:39.899924 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:39.899757 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade53f57-ad06-4ce6-aacf-8bcd0771e61e-samples-operator-tls podName:ade53f57-ad06-4ce6-aacf-8bcd0771e61e nodeName:}" failed. No retries permitted until 2026-09-08 02:22:40.39974355 +0000 UTC m=+55.931388397 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ade53f57-ad06-4ce6-aacf-8bcd0771e61e-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-t257k" (UID: "ade53f57-ad06-4ce6-aacf-8bcd0771e61e") : secret "samples-operator-tls" not found Sep 08 02:22:39.899924 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.899798 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqv5c\" (UniqueName: \"kubernetes.io/projected/55e533a6-3267-4b11-966b-677e70829961-kube-api-access-kqv5c\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:39.900088 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.900071 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/61eceefd-089d-4d1f-9088-1793fd4fe32d-config\") pod \"service-ca-operator-859f94c4b-drqqr\" (UID: \"61eceefd-089d-4d1f-9088-1793fd4fe32d\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-drqqr" Sep 08 02:22:39.902077 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.902054 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61eceefd-089d-4d1f-9088-1793fd4fe32d-serving-cert\") pod \"service-ca-operator-859f94c4b-drqqr\" (UID: \"61eceefd-089d-4d1f-9088-1793fd4fe32d\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-drqqr" Sep 08 02:22:39.902391 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.902374 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5d6c5474d8-9wrc6"] Sep 08 02:22:39.912291 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.912269 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njc5l\" (UniqueName: \"kubernetes.io/projected/ade53f57-ad06-4ce6-aacf-8bcd0771e61e-kube-api-access-njc5l\") pod \"cluster-samples-operator-7cdcfcf76c-t257k\" (UID: \"ade53f57-ad06-4ce6-aacf-8bcd0771e61e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-t257k" Sep 08 02:22:39.912558 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.912539 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2fml\" (UniqueName: \"kubernetes.io/projected/61eceefd-089d-4d1f-9088-1793fd4fe32d-kube-api-access-s2fml\") pod \"service-ca-operator-859f94c4b-drqqr\" (UID: \"61eceefd-089d-4d1f-9088-1793fd4fe32d\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-drqqr" Sep 08 02:22:39.986523 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:39.986440 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" Sep 08 02:22:40.000653 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:40.000630 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-default-certificate\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:40.000764 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:40.000687 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55e533a6-3267-4b11-966b-677e70829961-service-ca-bundle\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:40.000764 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:40.000713 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-metrics-certs\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:40.000854 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:40.000805 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-stats-auth\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:40.000854 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:40.000846 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqv5c\" (UniqueName: \"kubernetes.io/projected/55e533a6-3267-4b11-966b-677e70829961-kube-api-access-kqv5c\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:40.000952 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:40.000936 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/55e533a6-3267-4b11-966b-677e70829961-service-ca-bundle podName:55e533a6-3267-4b11-966b-677e70829961 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:40.500912883 +0000 UTC m=+56.032557725 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/55e533a6-3267-4b11-966b-677e70829961-service-ca-bundle") pod "router-default-5d6c5474d8-9wrc6" (UID: "55e533a6-3267-4b11-966b-677e70829961") : configmap references non-existent config key: service-ca.crt Sep 08 02:22:40.001014 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:40.000983 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 02:22:40.001066 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:40.001026 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-metrics-certs podName:55e533a6-3267-4b11-966b-677e70829961 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:40.50101337 +0000 UTC m=+56.032658231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-metrics-certs") pod "router-default-5d6c5474d8-9wrc6" (UID: "55e533a6-3267-4b11-966b-677e70829961") : secret "router-metrics-certs-default" not found Sep 08 02:22:40.002922 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:40.002899 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-default-certificate\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:40.002999 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:40.002970 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-stats-auth\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:40.009568 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:40.009546 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqv5c\" (UniqueName: \"kubernetes.io/projected/55e533a6-3267-4b11-966b-677e70829961-kube-api-access-kqv5c\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:40.092385 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:40.092352 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-drqqr" Sep 08 02:22:40.252026 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:40.251954 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-2hccm"] Sep 08 02:22:40.254573 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:22:40.254551 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod312a0520_e0c1_45b6_a9ac_e9f0b54df733.slice/crio-228f85bb83cbba42f27e5acde1d67cd2c8bed8813b05090e677b835422a4b341 WatchSource:0}: Error finding container 228f85bb83cbba42f27e5acde1d67cd2c8bed8813b05090e677b835422a4b341: Status 404 returned error can't find the container with id 228f85bb83cbba42f27e5acde1d67cd2c8bed8813b05090e677b835422a4b341 Sep 08 02:22:40.350138 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:40.350109 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" event={"ID":"312a0520-e0c1-45b6-a9ac-e9f0b54df733","Type":"ContainerStarted","Data":"228f85bb83cbba42f27e5acde1d67cd2c8bed8813b05090e677b835422a4b341"} Sep 08 02:22:40.356793 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:40.356768 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-drqqr"] Sep 08 02:22:40.359660 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:22:40.359640 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61eceefd_089d_4d1f_9088_1793fd4fe32d.slice/crio-8c067679481adb403bf2bdb3789137f6d6a26984d9e0109149199174488514d0 WatchSource:0}: Error finding container 8c067679481adb403bf2bdb3789137f6d6a26984d9e0109149199174488514d0: Status 404 returned error can't find the container with id 8c067679481adb403bf2bdb3789137f6d6a26984d9e0109149199174488514d0 Sep 08 02:22:40.404290 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:40.404265 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ade53f57-ad06-4ce6-aacf-8bcd0771e61e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-t257k\" (UID: \"ade53f57-ad06-4ce6-aacf-8bcd0771e61e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-t257k" Sep 08 02:22:40.404400 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:40.404388 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 08 02:22:40.404457 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:40.404441 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade53f57-ad06-4ce6-aacf-8bcd0771e61e-samples-operator-tls podName:ade53f57-ad06-4ce6-aacf-8bcd0771e61e nodeName:}" failed. No retries permitted until 2026-09-08 02:22:41.404427577 +0000 UTC m=+56.936072419 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ade53f57-ad06-4ce6-aacf-8bcd0771e61e-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-t257k" (UID: "ade53f57-ad06-4ce6-aacf-8bcd0771e61e") : secret "samples-operator-tls" not found Sep 08 02:22:40.504733 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:40.504663 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55e533a6-3267-4b11-966b-677e70829961-service-ca-bundle\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:40.504733 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:40.504706 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-metrics-certs\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:40.504862 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:40.504795 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 02:22:40.504862 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:40.504824 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/55e533a6-3267-4b11-966b-677e70829961-service-ca-bundle podName:55e533a6-3267-4b11-966b-677e70829961 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:41.504808142 +0000 UTC m=+57.036452990 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/55e533a6-3267-4b11-966b-677e70829961-service-ca-bundle") pod "router-default-5d6c5474d8-9wrc6" (UID: "55e533a6-3267-4b11-966b-677e70829961") : configmap references non-existent config key: service-ca.crt Sep 08 02:22:40.504933 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:40.504906 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-metrics-certs podName:55e533a6-3267-4b11-966b-677e70829961 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:41.504899048 +0000 UTC m=+57.036543891 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-metrics-certs") pod "router-default-5d6c5474d8-9wrc6" (UID: "55e533a6-3267-4b11-966b-677e70829961") : secret "router-metrics-certs-default" not found Sep 08 02:22:41.354033 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:41.353986 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-drqqr" event={"ID":"61eceefd-089d-4d1f-9088-1793fd4fe32d","Type":"ContainerStarted","Data":"8c067679481adb403bf2bdb3789137f6d6a26984d9e0109149199174488514d0"} Sep 08 02:22:41.412841 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:41.412785 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ade53f57-ad06-4ce6-aacf-8bcd0771e61e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-t257k\" (UID: \"ade53f57-ad06-4ce6-aacf-8bcd0771e61e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-t257k" Sep 08 02:22:41.413006 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:41.412963 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 08 02:22:41.413071 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:41.413040 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade53f57-ad06-4ce6-aacf-8bcd0771e61e-samples-operator-tls podName:ade53f57-ad06-4ce6-aacf-8bcd0771e61e nodeName:}" failed. No retries permitted until 2026-09-08 02:22:43.413020165 +0000 UTC m=+58.944665029 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ade53f57-ad06-4ce6-aacf-8bcd0771e61e-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-t257k" (UID: "ade53f57-ad06-4ce6-aacf-8bcd0771e61e") : secret "samples-operator-tls" not found Sep 08 02:22:41.514180 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:41.514140 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55e533a6-3267-4b11-966b-677e70829961-service-ca-bundle\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:41.514351 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:41.514190 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-metrics-certs\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:41.514351 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:41.514332 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/55e533a6-3267-4b11-966b-677e70829961-service-ca-bundle podName:55e533a6-3267-4b11-966b-677e70829961 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:43.514312169 +0000 UTC m=+59.045957033 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/55e533a6-3267-4b11-966b-677e70829961-service-ca-bundle") pod "router-default-5d6c5474d8-9wrc6" (UID: "55e533a6-3267-4b11-966b-677e70829961") : configmap references non-existent config key: service-ca.crt Sep 08 02:22:41.514496 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:41.514388 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 02:22:41.514496 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:41.514446 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-metrics-certs podName:55e533a6-3267-4b11-966b-677e70829961 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:43.51442804 +0000 UTC m=+59.046072884 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-metrics-certs") pod "router-default-5d6c5474d8-9wrc6" (UID: "55e533a6-3267-4b11-966b-677e70829961") : secret "router-metrics-certs-default" not found Sep 08 02:22:43.360916 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:43.360877 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" event={"ID":"312a0520-e0c1-45b6-a9ac-e9f0b54df733","Type":"ContainerStarted","Data":"bcd5af9c0dba13629008983e08bacc0dbfa3461f7f4670fb0202f61904364c11"} Sep 08 02:22:43.362379 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:43.362350 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-drqqr" event={"ID":"61eceefd-089d-4d1f-9088-1793fd4fe32d","Type":"ContainerStarted","Data":"c21792e81e8d3d3c673f451273c9638c443535a0f609f0f200da8ecb2a44fa10"} Sep 08 02:22:43.386007 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:43.383219 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" podStartSLOduration=2.013419637 podStartE2EDuration="4.38320082s" totalImagesPullingTime="2.369781183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:22:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:22:43.38170819 +0000 UTC m=+58.913353058" watchObservedRunningTime="2026-09-08 02:22:43.38320082 +0000 UTC m=+58.914845687" Sep 08 02:22:43.402294 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:43.402237 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-drqqr" podStartSLOduration=2.134728895 podStartE2EDuration="4.402215282s" totalImagesPullingTime="2.267486387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:22:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:22:43.401065616 +0000 UTC m=+58.932710481" watchObservedRunningTime="2026-09-08 02:22:43.402215282 +0000 UTC m=+58.933860147" Sep 08 02:22:43.431441 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:43.431209 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ade53f57-ad06-4ce6-aacf-8bcd0771e61e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-t257k\" (UID: \"ade53f57-ad06-4ce6-aacf-8bcd0771e61e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-t257k" Sep 08 02:22:43.431602 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:43.431487 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 08 02:22:43.431602 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:43.431541 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade53f57-ad06-4ce6-aacf-8bcd0771e61e-samples-operator-tls podName:ade53f57-ad06-4ce6-aacf-8bcd0771e61e nodeName:}" failed. No retries permitted until 2026-09-08 02:22:47.431527074 +0000 UTC m=+62.963171918 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ade53f57-ad06-4ce6-aacf-8bcd0771e61e-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-t257k" (UID: "ade53f57-ad06-4ce6-aacf-8bcd0771e61e") : secret "samples-operator-tls" not found Sep 08 02:22:43.531877 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:43.531835 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55e533a6-3267-4b11-966b-677e70829961-service-ca-bundle\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:43.531877 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:43.531873 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-metrics-certs\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:43.532106 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:43.531977 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 02:22:43.532106 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:43.532007 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/55e533a6-3267-4b11-966b-677e70829961-service-ca-bundle podName:55e533a6-3267-4b11-966b-677e70829961 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:47.531988259 +0000 UTC m=+63.063633122 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/55e533a6-3267-4b11-966b-677e70829961-service-ca-bundle") pod "router-default-5d6c5474d8-9wrc6" (UID: "55e533a6-3267-4b11-966b-677e70829961") : configmap references non-existent config key: service-ca.crt Sep 08 02:22:43.532106 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:43.532027 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-metrics-certs podName:55e533a6-3267-4b11-966b-677e70829961 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:47.532019936 +0000 UTC m=+63.063664779 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-metrics-certs") pod "router-default-5d6c5474d8-9wrc6" (UID: "55e533a6-3267-4b11-966b-677e70829961") : secret "router-metrics-certs-default" not found Sep 08 02:22:43.905381 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:43.905345 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lg2vn"] Sep 08 02:22:43.936136 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:43.936099 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lg2vn"] Sep 08 02:22:43.936305 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:43.936223 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:43.939234 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:43.939203 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 08 02:22:43.939349 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:43.939262 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 08 02:22:43.939349 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:43.939300 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nngdf\"" Sep 08 02:22:44.035819 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.035774 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/321393ae-48e7-4aab-b619-9f176576c695-crio-socket\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:44.036042 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.035845 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/321393ae-48e7-4aab-b619-9f176576c695-data-volume\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:44.036042 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.035864 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/321393ae-48e7-4aab-b619-9f176576c695-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:44.036042 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.035881 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbl9b\" (UniqueName: \"kubernetes.io/projected/321393ae-48e7-4aab-b619-9f176576c695-kube-api-access-dbl9b\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:44.036042 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.035989 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/321393ae-48e7-4aab-b619-9f176576c695-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:44.136655 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.136612 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/321393ae-48e7-4aab-b619-9f176576c695-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:44.136855 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.136678 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/321393ae-48e7-4aab-b619-9f176576c695-crio-socket\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:44.136855 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.136713 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/321393ae-48e7-4aab-b619-9f176576c695-data-volume\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:44.136855 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.136730 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/321393ae-48e7-4aab-b619-9f176576c695-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:44.136855 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:44.136731 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 02:22:44.136855 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.136745 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbl9b\" (UniqueName: \"kubernetes.io/projected/321393ae-48e7-4aab-b619-9f176576c695-kube-api-access-dbl9b\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:44.136855 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:44.136797 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/321393ae-48e7-4aab-b619-9f176576c695-kube-rbac-proxy-cm podName:321393ae-48e7-4aab-b619-9f176576c695 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:44.636780774 +0000 UTC m=+60.168425617 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/321393ae-48e7-4aab-b619-9f176576c695-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lg2vn" (UID: "321393ae-48e7-4aab-b619-9f176576c695") : configmap "kube-rbac-proxy" not found Sep 08 02:22:44.137153 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:44.136865 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 02:22:44.137153 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:44.136917 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/321393ae-48e7-4aab-b619-9f176576c695-insights-runtime-extractor-tls podName:321393ae-48e7-4aab-b619-9f176576c695 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:44.63690241 +0000 UTC m=+60.168547269 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/321393ae-48e7-4aab-b619-9f176576c695-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lg2vn" (UID: "321393ae-48e7-4aab-b619-9f176576c695") : secret "insights-runtime-extractor-tls" not found Sep 08 02:22:44.137153 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.136866 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/321393ae-48e7-4aab-b619-9f176576c695-crio-socket\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:44.137153 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.137127 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/321393ae-48e7-4aab-b619-9f176576c695-data-volume\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:44.147877 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.147849 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbl9b\" (UniqueName: \"kubernetes.io/projected/321393ae-48e7-4aab-b619-9f176576c695-kube-api-access-dbl9b\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:44.558398 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.558357 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-tp6v9"] Sep 08 02:22:44.579861 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.579831 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-tp6v9"] Sep 08 02:22:44.580013 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.579926 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-tp6v9" Sep 08 02:22:44.582548 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.582520 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Sep 08 02:22:44.582709 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.582690 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Sep 08 02:22:44.583897 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.583875 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-ht8tl\"" Sep 08 02:22:44.641395 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.641332 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/321393ae-48e7-4aab-b619-9f176576c695-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:44.641594 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:44.641450 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 02:22:44.641594 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.641455 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls2zn\" (UniqueName: \"kubernetes.io/projected/d7b07bfb-8a8b-4404-a9c7-13762299a74f-kube-api-access-ls2zn\") pod \"migrator-fdbb8fdff-tp6v9\" (UID: \"d7b07bfb-8a8b-4404-a9c7-13762299a74f\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-tp6v9" Sep 08 02:22:44.641594 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:44.641528 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/321393ae-48e7-4aab-b619-9f176576c695-kube-rbac-proxy-cm podName:321393ae-48e7-4aab-b619-9f176576c695 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:45.64151117 +0000 UTC m=+61.173156013 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/321393ae-48e7-4aab-b619-9f176576c695-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lg2vn" (UID: "321393ae-48e7-4aab-b619-9f176576c695") : configmap "kube-rbac-proxy" not found Sep 08 02:22:44.641746 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.641604 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/321393ae-48e7-4aab-b619-9f176576c695-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:44.641746 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:44.641740 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 02:22:44.641820 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:44.641788 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/321393ae-48e7-4aab-b619-9f176576c695-insights-runtime-extractor-tls podName:321393ae-48e7-4aab-b619-9f176576c695 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:45.641774461 +0000 UTC m=+61.173419318 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/321393ae-48e7-4aab-b619-9f176576c695-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lg2vn" (UID: "321393ae-48e7-4aab-b619-9f176576c695") : secret "insights-runtime-extractor-tls" not found Sep 08 02:22:44.742853 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.742818 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls2zn\" (UniqueName: \"kubernetes.io/projected/d7b07bfb-8a8b-4404-a9c7-13762299a74f-kube-api-access-ls2zn\") pod \"migrator-fdbb8fdff-tp6v9\" (UID: \"d7b07bfb-8a8b-4404-a9c7-13762299a74f\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-tp6v9" Sep 08 02:22:44.752224 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.752192 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls2zn\" (UniqueName: \"kubernetes.io/projected/d7b07bfb-8a8b-4404-a9c7-13762299a74f-kube-api-access-ls2zn\") pod \"migrator-fdbb8fdff-tp6v9\" (UID: \"d7b07bfb-8a8b-4404-a9c7-13762299a74f\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-tp6v9" Sep 08 02:22:44.891640 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.891608 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-ht8tl\"" Sep 08 02:22:44.899635 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:44.899604 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-tp6v9" Sep 08 02:22:45.195725 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:45.195650 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-tp6v9"] Sep 08 02:22:45.199034 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:22:45.199003 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7b07bfb_8a8b_4404_a9c7_13762299a74f.slice/crio-f64a7a72ed3412f5dac5696947046f4d21e9e8dce8721fd6b334d039c179d52b WatchSource:0}: Error finding container f64a7a72ed3412f5dac5696947046f4d21e9e8dce8721fd6b334d039c179d52b: Status 404 returned error can't find the container with id f64a7a72ed3412f5dac5696947046f4d21e9e8dce8721fd6b334d039c179d52b Sep 08 02:22:45.284824 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:45.284796 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wcjds" Sep 08 02:22:45.368715 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:45.368678 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-tp6v9" event={"ID":"d7b07bfb-8a8b-4404-a9c7-13762299a74f","Type":"ContainerStarted","Data":"f64a7a72ed3412f5dac5696947046f4d21e9e8dce8721fd6b334d039c179d52b"} Sep 08 02:22:45.650014 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:45.649980 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/321393ae-48e7-4aab-b619-9f176576c695-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:45.650380 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:45.650084 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/321393ae-48e7-4aab-b619-9f176576c695-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:45.650380 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:45.650135 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 02:22:45.650380 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:45.650194 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 02:22:45.650380 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:45.650203 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/321393ae-48e7-4aab-b619-9f176576c695-insights-runtime-extractor-tls podName:321393ae-48e7-4aab-b619-9f176576c695 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:47.650186407 +0000 UTC m=+63.181831250 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/321393ae-48e7-4aab-b619-9f176576c695-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lg2vn" (UID: "321393ae-48e7-4aab-b619-9f176576c695") : secret "insights-runtime-extractor-tls" not found Sep 08 02:22:45.650380 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:45.650251 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/321393ae-48e7-4aab-b619-9f176576c695-kube-rbac-proxy-cm podName:321393ae-48e7-4aab-b619-9f176576c695 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:47.650238882 +0000 UTC m=+63.181883729 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/321393ae-48e7-4aab-b619-9f176576c695-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lg2vn" (UID: "321393ae-48e7-4aab-b619-9f176576c695") : configmap "kube-rbac-proxy" not found Sep 08 02:22:45.673214 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:45.673194 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-66vxb_185c152f-08c1-48bf-9d40-f61ee106ff12/dns-node-resolver/0.log" Sep 08 02:22:46.869983 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:46.869948 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-c9hcb"] Sep 08 02:22:46.871866 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:46.871849 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-c9hcb" Sep 08 02:22:46.874983 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:46.874961 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 08 02:22:46.875126 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:46.875000 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 08 02:22:46.876224 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:46.876201 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-ffrqh\"" Sep 08 02:22:46.876326 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:46.876244 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 08 02:22:46.876326 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:46.876297 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 08 02:22:46.884117 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:46.884079 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-c9hcb"] Sep 08 02:22:46.960020 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:46.959978 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68bcw\" (UniqueName: \"kubernetes.io/projected/b272a25c-4296-4750-8e79-10a78ca7c8b7-kube-api-access-68bcw\") pod \"service-ca-75d99c484-c9hcb\" (UID: \"b272a25c-4296-4750-8e79-10a78ca7c8b7\") " pod="openshift-service-ca/service-ca-75d99c484-c9hcb" Sep 08 02:22:46.960020 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:46.960026 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b272a25c-4296-4750-8e79-10a78ca7c8b7-signing-cabundle\") pod \"service-ca-75d99c484-c9hcb\" (UID: \"b272a25c-4296-4750-8e79-10a78ca7c8b7\") " pod="openshift-service-ca/service-ca-75d99c484-c9hcb" Sep 08 02:22:46.960264 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:46.960118 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b272a25c-4296-4750-8e79-10a78ca7c8b7-signing-key\") pod \"service-ca-75d99c484-c9hcb\" (UID: \"b272a25c-4296-4750-8e79-10a78ca7c8b7\") " pod="openshift-service-ca/service-ca-75d99c484-c9hcb" Sep 08 02:22:47.060785 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:47.060748 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68bcw\" (UniqueName: \"kubernetes.io/projected/b272a25c-4296-4750-8e79-10a78ca7c8b7-kube-api-access-68bcw\") pod \"service-ca-75d99c484-c9hcb\" (UID: \"b272a25c-4296-4750-8e79-10a78ca7c8b7\") " pod="openshift-service-ca/service-ca-75d99c484-c9hcb" Sep 08 02:22:47.060785 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:47.060789 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b272a25c-4296-4750-8e79-10a78ca7c8b7-signing-cabundle\") pod \"service-ca-75d99c484-c9hcb\" (UID: \"b272a25c-4296-4750-8e79-10a78ca7c8b7\") " pod="openshift-service-ca/service-ca-75d99c484-c9hcb" Sep 08 02:22:47.060972 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:47.060828 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b272a25c-4296-4750-8e79-10a78ca7c8b7-signing-key\") pod \"service-ca-75d99c484-c9hcb\" (UID: \"b272a25c-4296-4750-8e79-10a78ca7c8b7\") " pod="openshift-service-ca/service-ca-75d99c484-c9hcb" Sep 08 02:22:47.061366 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:47.061339 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b272a25c-4296-4750-8e79-10a78ca7c8b7-signing-cabundle\") pod \"service-ca-75d99c484-c9hcb\" (UID: \"b272a25c-4296-4750-8e79-10a78ca7c8b7\") " pod="openshift-service-ca/service-ca-75d99c484-c9hcb" Sep 08 02:22:47.063124 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:47.063104 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b272a25c-4296-4750-8e79-10a78ca7c8b7-signing-key\") pod \"service-ca-75d99c484-c9hcb\" (UID: \"b272a25c-4296-4750-8e79-10a78ca7c8b7\") " pod="openshift-service-ca/service-ca-75d99c484-c9hcb" Sep 08 02:22:47.072104 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:47.072081 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68bcw\" (UniqueName: \"kubernetes.io/projected/b272a25c-4296-4750-8e79-10a78ca7c8b7-kube-api-access-68bcw\") pod \"service-ca-75d99c484-c9hcb\" (UID: \"b272a25c-4296-4750-8e79-10a78ca7c8b7\") " pod="openshift-service-ca/service-ca-75d99c484-c9hcb" Sep 08 02:22:47.083312 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:47.083293 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vn7fg_6fe0dc65-7890-405f-9c55-755957407695/node-ca/0.log" Sep 08 02:22:47.180355 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:47.180277 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-c9hcb" Sep 08 02:22:47.374380 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:47.374343 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-tp6v9" event={"ID":"d7b07bfb-8a8b-4404-a9c7-13762299a74f","Type":"ContainerStarted","Data":"8b3282de4d74806cf594eead3e1763a9bde6f9ee2c182bd65d7d92dd4dc19959"} Sep 08 02:22:47.374555 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:47.374413 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-tp6v9" event={"ID":"d7b07bfb-8a8b-4404-a9c7-13762299a74f","Type":"ContainerStarted","Data":"3c9b4445a50db5bee27fa3b297e79be1422f06d60c4f9169725c3cf52134d40c"} Sep 08 02:22:47.396351 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:47.394562 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-tp6v9" podStartSLOduration=2.231917936 podStartE2EDuration="3.394542674s" totalImagesPullingTime="1.162624738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:22:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:22:47.393009467 +0000 UTC m=+62.924654334" watchObservedRunningTime="2026-09-08 02:22:47.394542674 +0000 UTC m=+62.926187551" Sep 08 02:22:47.446123 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:47.446051 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-c9hcb"] Sep 08 02:22:47.448732 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:22:47.448707 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb272a25c_4296_4750_8e79_10a78ca7c8b7.slice/crio-de32f82e0b4e953d02f24ea8034f6796cf97110ea91cf1f86454d8c9331c1493 WatchSource:0}: Error finding container de32f82e0b4e953d02f24ea8034f6796cf97110ea91cf1f86454d8c9331c1493: Status 404 returned error can't find the container with id de32f82e0b4e953d02f24ea8034f6796cf97110ea91cf1f86454d8c9331c1493 Sep 08 02:22:47.463437 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:47.463414 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ade53f57-ad06-4ce6-aacf-8bcd0771e61e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-t257k\" (UID: \"ade53f57-ad06-4ce6-aacf-8bcd0771e61e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-t257k" Sep 08 02:22:47.463549 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:47.463531 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 08 02:22:47.463610 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:47.463600 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade53f57-ad06-4ce6-aacf-8bcd0771e61e-samples-operator-tls podName:ade53f57-ad06-4ce6-aacf-8bcd0771e61e nodeName:}" failed. No retries permitted until 2026-09-08 02:22:55.463582544 +0000 UTC m=+70.995227386 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ade53f57-ad06-4ce6-aacf-8bcd0771e61e-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-t257k" (UID: "ade53f57-ad06-4ce6-aacf-8bcd0771e61e") : secret "samples-operator-tls" not found Sep 08 02:22:47.564550 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:47.564508 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55e533a6-3267-4b11-966b-677e70829961-service-ca-bundle\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:47.564753 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:47.564613 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-metrics-certs\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:47.564753 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:47.564694 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/55e533a6-3267-4b11-966b-677e70829961-service-ca-bundle podName:55e533a6-3267-4b11-966b-677e70829961 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:55.56467527 +0000 UTC m=+71.096320143 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/55e533a6-3267-4b11-966b-677e70829961-service-ca-bundle") pod "router-default-5d6c5474d8-9wrc6" (UID: "55e533a6-3267-4b11-966b-677e70829961") : configmap references non-existent config key: service-ca.crt Sep 08 02:22:47.564753 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:47.564734 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 02:22:47.564921 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:47.564783 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-metrics-certs podName:55e533a6-3267-4b11-966b-677e70829961 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:55.564770568 +0000 UTC m=+71.096415410 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-metrics-certs") pod "router-default-5d6c5474d8-9wrc6" (UID: "55e533a6-3267-4b11-966b-677e70829961") : secret "router-metrics-certs-default" not found Sep 08 02:22:47.665591 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:47.665549 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/321393ae-48e7-4aab-b619-9f176576c695-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:47.665774 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:47.665604 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/321393ae-48e7-4aab-b619-9f176576c695-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:47.665774 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:47.665671 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 02:22:47.665774 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:47.665704 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 02:22:47.665774 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:47.665716 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/321393ae-48e7-4aab-b619-9f176576c695-kube-rbac-proxy-cm podName:321393ae-48e7-4aab-b619-9f176576c695 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:51.665703477 +0000 UTC m=+67.197348320 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/321393ae-48e7-4aab-b619-9f176576c695-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lg2vn" (UID: "321393ae-48e7-4aab-b619-9f176576c695") : configmap "kube-rbac-proxy" not found Sep 08 02:22:47.665983 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:47.665779 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/321393ae-48e7-4aab-b619-9f176576c695-insights-runtime-extractor-tls podName:321393ae-48e7-4aab-b619-9f176576c695 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:51.665763156 +0000 UTC m=+67.197408000 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/321393ae-48e7-4aab-b619-9f176576c695-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lg2vn" (UID: "321393ae-48e7-4aab-b619-9f176576c695") : secret "insights-runtime-extractor-tls" not found Sep 08 02:22:47.874177 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:47.874133 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p6qlw_aeddc1eb-1d10-4d91-a627-6d0cae0ad53b/serve-healthcheck-canary/0.log" Sep 08 02:22:48.380000 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:48.379963 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-c9hcb" event={"ID":"b272a25c-4296-4750-8e79-10a78ca7c8b7","Type":"ContainerStarted","Data":"1467b106930ca55413b34e07a9ada4d052230ac8aaf565aa719b6eac47f59852"} Sep 08 02:22:48.380165 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:48.380021 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-c9hcb" event={"ID":"b272a25c-4296-4750-8e79-10a78ca7c8b7","Type":"ContainerStarted","Data":"de32f82e0b4e953d02f24ea8034f6796cf97110ea91cf1f86454d8c9331c1493"} Sep 08 02:22:48.401349 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:48.401306 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-c9hcb" podStartSLOduration=2.401293087 podStartE2EDuration="2.401293087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:22:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:22:48.400162108 +0000 UTC m=+63.931806972" watchObservedRunningTime="2026-09-08 02:22:48.401293087 +0000 UTC m=+63.932937978" Sep 08 02:22:50.591069 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:50.591028 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls\") pod \"dns-default-qhkj2\" (UID: \"382809a0-5fcd-4635-8ab5-f7e83672b1a5\") " pod="openshift-dns/dns-default-qhkj2" Sep 08 02:22:50.591452 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:50.591170 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 02:22:50.591452 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:50.591234 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls podName:382809a0-5fcd-4635-8ab5-f7e83672b1a5 nodeName:}" failed. No retries permitted until 2026-09-08 02:23:22.591218334 +0000 UTC m=+98.122863177 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls") pod "dns-default-qhkj2" (UID: "382809a0-5fcd-4635-8ab5-f7e83672b1a5") : secret "dns-default-metrics-tls" not found Sep 08 02:22:50.692367 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:50.692334 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret\") pod \"global-pull-secret-syncer-pmw7v\" (UID: \"0af6038b-8bad-4d6a-9478-083edfbc974a\") " pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:22:50.695554 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:50.695536 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 08 02:22:50.705360 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:50.705332 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0af6038b-8bad-4d6a-9478-083edfbc974a-original-pull-secret\") pod \"global-pull-secret-syncer-pmw7v\" (UID: \"0af6038b-8bad-4d6a-9478-083edfbc974a\") " pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:22:50.792736 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:50.792708 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2987\" (UniqueName: \"kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987\") pod \"network-check-target-c8fz9\" (UID: \"fe2f6a1b-8c73-4c40-8525-d1329e889f46\") " pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:22:50.792881 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:50.792763 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs\") pod \"network-metrics-daemon-mt4gl\" (UID: \"ec8f0aba-d0c2-4d45-b25d-84e2790d49ec\") " pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:22:50.795767 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:50.795748 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 08 02:22:50.795861 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:50.795772 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 08 02:22:50.803452 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:50.803430 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 08 02:22:50.803550 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:50.803509 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs podName:ec8f0aba-d0c2-4d45-b25d-84e2790d49ec nodeName:}" failed. No retries permitted until 2026-09-08 02:23:54.803488675 +0000 UTC m=+130.335133525 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs") pod "network-metrics-daemon-mt4gl" (UID: "ec8f0aba-d0c2-4d45-b25d-84e2790d49ec") : secret "metrics-daemon-secret" not found Sep 08 02:22:50.806278 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:50.806263 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 08 02:22:50.816503 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:50.816479 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2987\" (UniqueName: \"kubernetes.io/projected/fe2f6a1b-8c73-4c40-8525-d1329e889f46-kube-api-access-w2987\") pod \"network-check-target-c8fz9\" (UID: \"fe2f6a1b-8c73-4c40-8525-d1329e889f46\") " pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:22:50.827057 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:50.827040 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m9nsl\"" Sep 08 02:22:50.831474 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:50.831451 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pmw7v" Sep 08 02:22:50.834586 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:50.834568 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:22:51.144634 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:51.144548 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pmw7v"] Sep 08 02:22:51.148619 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:22:51.148594 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0af6038b_8bad_4d6a_9478_083edfbc974a.slice/crio-ce5fbba9388a0ffb144e0012599924f3185a9bc57adcfac3b6e340d8cb59933a WatchSource:0}: Error finding container ce5fbba9388a0ffb144e0012599924f3185a9bc57adcfac3b6e340d8cb59933a: Status 404 returned error can't find the container with id ce5fbba9388a0ffb144e0012599924f3185a9bc57adcfac3b6e340d8cb59933a Sep 08 02:22:51.163366 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:51.163281 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c8fz9"] Sep 08 02:22:51.165681 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:22:51.165657 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe2f6a1b_8c73_4c40_8525_d1329e889f46.slice/crio-9e9591665cfa4c4f12e73f56b82c282d9c07d218761835488494454f219b886c WatchSource:0}: Error finding container 9e9591665cfa4c4f12e73f56b82c282d9c07d218761835488494454f219b886c: Status 404 returned error can't find the container with id 9e9591665cfa4c4f12e73f56b82c282d9c07d218761835488494454f219b886c Sep 08 02:22:51.399975 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:51.396140 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pmw7v" event={"ID":"0af6038b-8bad-4d6a-9478-083edfbc974a","Type":"ContainerStarted","Data":"ce5fbba9388a0ffb144e0012599924f3185a9bc57adcfac3b6e340d8cb59933a"} Sep 08 02:22:51.399975 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:51.398126 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c8fz9" event={"ID":"fe2f6a1b-8c73-4c40-8525-d1329e889f46","Type":"ContainerStarted","Data":"9e9591665cfa4c4f12e73f56b82c282d9c07d218761835488494454f219b886c"} Sep 08 02:22:51.701650 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:51.701560 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/321393ae-48e7-4aab-b619-9f176576c695-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:51.701650 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:51.701632 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/321393ae-48e7-4aab-b619-9f176576c695-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:51.702128 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:51.701740 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 02:22:51.702128 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:51.701809 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/321393ae-48e7-4aab-b619-9f176576c695-kube-rbac-proxy-cm podName:321393ae-48e7-4aab-b619-9f176576c695 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:59.701788365 +0000 UTC m=+75.233433210 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/321393ae-48e7-4aab-b619-9f176576c695-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lg2vn" (UID: "321393ae-48e7-4aab-b619-9f176576c695") : configmap "kube-rbac-proxy" not found Sep 08 02:22:51.702128 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:51.701826 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 02:22:51.702128 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:22:51.701891 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/321393ae-48e7-4aab-b619-9f176576c695-insights-runtime-extractor-tls podName:321393ae-48e7-4aab-b619-9f176576c695 nodeName:}" failed. No retries permitted until 2026-09-08 02:22:59.7018728 +0000 UTC m=+75.233517645 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/321393ae-48e7-4aab-b619-9f176576c695-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lg2vn" (UID: "321393ae-48e7-4aab-b619-9f176576c695") : secret "insights-runtime-extractor-tls" not found Sep 08 02:22:55.537202 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:55.537167 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ade53f57-ad06-4ce6-aacf-8bcd0771e61e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-t257k\" (UID: \"ade53f57-ad06-4ce6-aacf-8bcd0771e61e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-t257k" Sep 08 02:22:55.539838 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:55.539811 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ade53f57-ad06-4ce6-aacf-8bcd0771e61e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-t257k\" (UID: \"ade53f57-ad06-4ce6-aacf-8bcd0771e61e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-t257k" Sep 08 02:22:55.638052 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:55.638004 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55e533a6-3267-4b11-966b-677e70829961-service-ca-bundle\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:55.638250 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:55.638114 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-metrics-certs\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:55.638783 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:55.638753 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55e533a6-3267-4b11-966b-677e70829961-service-ca-bundle\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:55.640798 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:55.640772 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55e533a6-3267-4b11-966b-677e70829961-metrics-certs\") pod \"router-default-5d6c5474d8-9wrc6\" (UID: \"55e533a6-3267-4b11-966b-677e70829961\") " pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:55.688886 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:55.688860 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-r8hht\"" Sep 08 02:22:55.696603 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:55.696574 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-t257k" Sep 08 02:22:55.798238 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:55.798220 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-hqj47\"" Sep 08 02:22:55.804603 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:55.804577 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:56.060874 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:56.060762 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-t257k"] Sep 08 02:22:56.103170 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:56.103147 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5d6c5474d8-9wrc6"] Sep 08 02:22:56.105394 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:22:56.105371 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55e533a6_3267_4b11_966b_677e70829961.slice/crio-eaa71e83eb2333ff187f1440c2accdbb74e7b5e9a2597830c120fbce0015463d WatchSource:0}: Error finding container eaa71e83eb2333ff187f1440c2accdbb74e7b5e9a2597830c120fbce0015463d: Status 404 returned error can't find the container with id eaa71e83eb2333ff187f1440c2accdbb74e7b5e9a2597830c120fbce0015463d Sep 08 02:22:56.422266 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:56.422224 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c8fz9" event={"ID":"fe2f6a1b-8c73-4c40-8525-d1329e889f46","Type":"ContainerStarted","Data":"772a83ff4bdf7f351e7a1183f083975b630ebcb590fe0ce7ea627dea359727b8"} Sep 08 02:22:56.422435 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:56.422392 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:22:56.424191 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:56.424160 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pmw7v" event={"ID":"0af6038b-8bad-4d6a-9478-083edfbc974a","Type":"ContainerStarted","Data":"e16a9e67a0c380570433647dd0d531e87923082eb372170ccf99122979b1cc75"} Sep 08 02:22:56.425510 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:56.425484 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-t257k" event={"ID":"ade53f57-ad06-4ce6-aacf-8bcd0771e61e","Type":"ContainerStarted","Data":"d94c31fd68182501b64aa4e6c7b3ae778aa8b2cc6a47888919194acd327a901e"} Sep 08 02:22:56.426787 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:56.426762 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" event={"ID":"55e533a6-3267-4b11-966b-677e70829961","Type":"ContainerStarted","Data":"cdc2394f0ce199d3e872903828ed51ce08dcbe69b7e9f20721b3ad3a6aa74e7a"} Sep 08 02:22:56.426873 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:56.426804 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" event={"ID":"55e533a6-3267-4b11-966b-677e70829961","Type":"ContainerStarted","Data":"eaa71e83eb2333ff187f1440c2accdbb74e7b5e9a2597830c120fbce0015463d"} Sep 08 02:22:56.440107 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:56.440061 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-c8fz9" podStartSLOduration=66.810554637 podStartE2EDuration="1m11.44004922s" totalImagesPullingTime="4.629494583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:22:56.439037827 +0000 UTC m=+71.970682691" watchObservedRunningTime="2026-09-08 02:22:56.44004922 +0000 UTC m=+71.971694085" Sep 08 02:22:56.456565 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:56.456526 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pmw7v" podStartSLOduration=66.80271903 podStartE2EDuration="1m11.456513056s" totalImagesPullingTime="4.653794026s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 02:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:22:56.45603419 +0000 UTC m=+71.987679054" watchObservedRunningTime="2026-09-08 02:22:56.456513056 +0000 UTC m=+71.988157920" Sep 08 02:22:56.475997 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:56.475948 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" podStartSLOduration=17.475936254 podStartE2EDuration="17.475936254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:22:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:22:56.474766226 +0000 UTC m=+72.006411095" watchObservedRunningTime="2026-09-08 02:22:56.475936254 +0000 UTC m=+72.007581120" Sep 08 02:22:56.805557 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:56.805453 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:56.809320 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:56.809295 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:57.432653 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:57.432610 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:57.433496 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:57.433371 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5d6c5474d8-9wrc6" Sep 08 02:22:58.435908 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:58.435865 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-t257k" event={"ID":"ade53f57-ad06-4ce6-aacf-8bcd0771e61e","Type":"ContainerStarted","Data":"60d869281b1e01fc26a2cbfa255aa30ce6b870e9c4e66d7326d94c26c84c423b"} Sep 08 02:22:58.436296 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:58.435923 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-t257k" event={"ID":"ade53f57-ad06-4ce6-aacf-8bcd0771e61e","Type":"ContainerStarted","Data":"852174db43227d4eaffe1ddcaadbd56b5ec0042e2e28ab0b5d6459eafa7eef63"} Sep 08 02:22:58.457626 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:58.457576 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-t257k" podStartSLOduration=17.781517145 podStartE2EDuration="19.457560619s" totalImagesPullingTime="1.676043474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:22:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:22:58.456885343 +0000 UTC m=+73.988530208" watchObservedRunningTime="2026-09-08 02:22:58.457560619 +0000 UTC m=+73.989205483" Sep 08 02:22:59.776489 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:59.776419 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/321393ae-48e7-4aab-b619-9f176576c695-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:59.776962 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:59.776496 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/321393ae-48e7-4aab-b619-9f176576c695-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:59.777103 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:59.777081 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/321393ae-48e7-4aab-b619-9f176576c695-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:59.778653 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:59.778633 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/321393ae-48e7-4aab-b619-9f176576c695-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lg2vn\" (UID: \"321393ae-48e7-4aab-b619-9f176576c695\") " pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:22:59.848291 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:59.848262 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nngdf\"" Sep 08 02:22:59.856166 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:22:59.856148 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lg2vn" Sep 08 02:23:00.128849 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:00.128763 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lg2vn"] Sep 08 02:23:00.132423 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:23:00.132399 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod321393ae_48e7_4aab_b619_9f176576c695.slice/crio-fad5ee72f78490e7f0a5d12135bfb65d9affea766a7f9b81c53df42f854f7499 WatchSource:0}: Error finding container fad5ee72f78490e7f0a5d12135bfb65d9affea766a7f9b81c53df42f854f7499: Status 404 returned error can't find the container with id fad5ee72f78490e7f0a5d12135bfb65d9affea766a7f9b81c53df42f854f7499 Sep 08 02:23:00.444495 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:00.444400 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lg2vn" event={"ID":"321393ae-48e7-4aab-b619-9f176576c695","Type":"ContainerStarted","Data":"1da2552241e0b245143a669e85ee25124d6f63eab998f4806a12e701d0c9481b"} Sep 08 02:23:00.444495 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:00.444470 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lg2vn" event={"ID":"321393ae-48e7-4aab-b619-9f176576c695","Type":"ContainerStarted","Data":"fad5ee72f78490e7f0a5d12135bfb65d9affea766a7f9b81c53df42f854f7499"} Sep 08 02:23:01.450599 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:01.450555 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lg2vn" event={"ID":"321393ae-48e7-4aab-b619-9f176576c695","Type":"ContainerStarted","Data":"8dd66a06f1b6ba3f253df106811bce27ad6fe2ec80348b45cc953c46788c451d"} Sep 08 02:23:02.456204 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:02.456169 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lg2vn" event={"ID":"321393ae-48e7-4aab-b619-9f176576c695","Type":"ContainerStarted","Data":"6addb41cb70efc34fb717fca29411dde11b6a47948f567890af0fb174eb42f78"} Sep 08 02:23:02.477540 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:02.477490 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lg2vn" podStartSLOduration=17.464170331 podStartE2EDuration="19.477451296s" totalImagesPullingTime="2.013280965s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 02:22:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:23:02.475763659 +0000 UTC m=+78.007408523" watchObservedRunningTime="2026-09-08 02:23:02.477451296 +0000 UTC m=+78.009096162" Sep 08 02:23:09.639190 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:09.639156 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-gc2gg"] Sep 08 02:23:09.643421 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:09.643404 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-gc2gg" Sep 08 02:23:09.647659 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:09.647638 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 08 02:23:09.647800 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:09.647780 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 08 02:23:09.647867 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:09.647848 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-75hm9\"" Sep 08 02:23:09.654852 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:09.654835 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-gc2gg"] Sep 08 02:23:09.751622 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:09.751545 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fslv8\" (UniqueName: \"kubernetes.io/projected/2d721641-4617-481e-8947-e5058100df47-kube-api-access-fslv8\") pod \"downloads-85d98c998f-gc2gg\" (UID: \"2d721641-4617-481e-8947-e5058100df47\") " pod="openshift-console/downloads-85d98c998f-gc2gg" Sep 08 02:23:09.852681 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:09.852643 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fslv8\" (UniqueName: \"kubernetes.io/projected/2d721641-4617-481e-8947-e5058100df47-kube-api-access-fslv8\") pod \"downloads-85d98c998f-gc2gg\" (UID: \"2d721641-4617-481e-8947-e5058100df47\") " pod="openshift-console/downloads-85d98c998f-gc2gg" Sep 08 02:23:09.899737 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:09.899708 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fslv8\" (UniqueName: \"kubernetes.io/projected/2d721641-4617-481e-8947-e5058100df47-kube-api-access-fslv8\") pod \"downloads-85d98c998f-gc2gg\" (UID: \"2d721641-4617-481e-8947-e5058100df47\") " pod="openshift-console/downloads-85d98c998f-gc2gg" Sep 08 02:23:09.951861 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:09.951832 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-gc2gg" Sep 08 02:23:10.231104 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:10.231072 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-gc2gg"] Sep 08 02:23:10.233718 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:23:10.233676 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d721641_4617_481e_8947_e5058100df47.slice/crio-d9303cdb16979a3b577eca3f0c64f77a2e6c36424b6092753fe4d47fb2640934 WatchSource:0}: Error finding container d9303cdb16979a3b577eca3f0c64f77a2e6c36424b6092753fe4d47fb2640934: Status 404 returned error can't find the container with id d9303cdb16979a3b577eca3f0c64f77a2e6c36424b6092753fe4d47fb2640934 Sep 08 02:23:10.482171 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:10.482094 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-gc2gg" event={"ID":"2d721641-4617-481e-8947-e5058100df47","Type":"ContainerStarted","Data":"d9303cdb16979a3b577eca3f0c64f77a2e6c36424b6092753fe4d47fb2640934"} Sep 08 02:23:14.246338 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:14.246301 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7zh78"] Sep 08 02:23:14.272433 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:14.272400 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7zh78"] Sep 08 02:23:14.272598 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:14.272542 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7zh78" Sep 08 02:23:14.276691 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:14.276593 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 08 02:23:14.276691 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:14.276591 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-bpl6n\"" Sep 08 02:23:14.386730 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:14.386692 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/045f37f9-6764-4abf-9545-bea7e4137bf5-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-7zh78\" (UID: \"045f37f9-6764-4abf-9545-bea7e4137bf5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7zh78" Sep 08 02:23:14.487944 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:14.487900 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/045f37f9-6764-4abf-9545-bea7e4137bf5-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-7zh78\" (UID: \"045f37f9-6764-4abf-9545-bea7e4137bf5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7zh78" Sep 08 02:23:14.490777 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:14.490746 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/045f37f9-6764-4abf-9545-bea7e4137bf5-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-7zh78\" (UID: \"045f37f9-6764-4abf-9545-bea7e4137bf5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7zh78" Sep 08 02:23:14.584296 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:14.584213 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7zh78" Sep 08 02:23:14.864137 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:14.864105 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7zh78"] Sep 08 02:23:14.866986 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:23:14.866960 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod045f37f9_6764_4abf_9545_bea7e4137bf5.slice/crio-d8042eaa40ea40ae3cfef43eda53429a5bade5bf1762fb81a2b8bbe5614b6a91 WatchSource:0}: Error finding container d8042eaa40ea40ae3cfef43eda53429a5bade5bf1762fb81a2b8bbe5614b6a91: Status 404 returned error can't find the container with id d8042eaa40ea40ae3cfef43eda53429a5bade5bf1762fb81a2b8bbe5614b6a91 Sep 08 02:23:15.503175 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:15.503136 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7zh78" event={"ID":"045f37f9-6764-4abf-9545-bea7e4137bf5","Type":"ContainerStarted","Data":"d8042eaa40ea40ae3cfef43eda53429a5bade5bf1762fb81a2b8bbe5614b6a91"} Sep 08 02:23:16.508400 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:16.508361 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7zh78" event={"ID":"045f37f9-6764-4abf-9545-bea7e4137bf5","Type":"ContainerStarted","Data":"1ee7e7b61e40fa63ca2e272f95c7fc516d577f8d9f9b721ae3656abf757e0153"} Sep 08 02:23:16.508876 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:16.508745 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7zh78" Sep 08 02:23:16.514219 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:16.514194 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7zh78" Sep 08 02:23:16.526851 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:16.526802 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7zh78" podStartSLOduration=1.488059345 podStartE2EDuration="2.526782769s" totalImagesPullingTime="1.038723424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:23:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:23:16.525820257 +0000 UTC m=+92.057465122" watchObservedRunningTime="2026-09-08 02:23:16.526782769 +0000 UTC m=+92.058427634" Sep 08 02:23:17.313399 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:17.313363 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-7qxms"] Sep 08 02:23:17.316438 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:17.316414 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-7qxms" Sep 08 02:23:17.319201 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:17.319178 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 08 02:23:17.320948 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:17.320533 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 08 02:23:17.320948 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:17.320562 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 08 02:23:17.320948 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:17.320594 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 08 02:23:17.320948 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:17.320542 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 08 02:23:17.320948 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:17.320868 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-zhdjl\"" Sep 08 02:23:17.326246 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:17.326203 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-7qxms"] Sep 08 02:23:17.411253 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:17.411219 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3deb3cf9-9352-4589-82bd-3ea0f048181a-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-7qxms\" (UID: \"3deb3cf9-9352-4589-82bd-3ea0f048181a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7qxms" Sep 08 02:23:17.411253 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:17.411266 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3deb3cf9-9352-4589-82bd-3ea0f048181a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-7qxms\" (UID: \"3deb3cf9-9352-4589-82bd-3ea0f048181a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7qxms" Sep 08 02:23:17.411567 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:17.411299 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74pk7\" (UniqueName: \"kubernetes.io/projected/3deb3cf9-9352-4589-82bd-3ea0f048181a-kube-api-access-74pk7\") pod \"prometheus-operator-6f4946db56-7qxms\" (UID: \"3deb3cf9-9352-4589-82bd-3ea0f048181a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7qxms" Sep 08 02:23:17.411567 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:17.411402 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3deb3cf9-9352-4589-82bd-3ea0f048181a-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-7qxms\" (UID: \"3deb3cf9-9352-4589-82bd-3ea0f048181a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7qxms" Sep 08 02:23:17.512428 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:17.512391 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3deb3cf9-9352-4589-82bd-3ea0f048181a-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-7qxms\" (UID: \"3deb3cf9-9352-4589-82bd-3ea0f048181a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7qxms" Sep 08 02:23:17.512923 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:17.512454 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3deb3cf9-9352-4589-82bd-3ea0f048181a-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-7qxms\" (UID: \"3deb3cf9-9352-4589-82bd-3ea0f048181a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7qxms" Sep 08 02:23:17.512923 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:17.512506 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3deb3cf9-9352-4589-82bd-3ea0f048181a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-7qxms\" (UID: \"3deb3cf9-9352-4589-82bd-3ea0f048181a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7qxms" Sep 08 02:23:17.512923 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:17.512542 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74pk7\" (UniqueName: \"kubernetes.io/projected/3deb3cf9-9352-4589-82bd-3ea0f048181a-kube-api-access-74pk7\") pod \"prometheus-operator-6f4946db56-7qxms\" (UID: \"3deb3cf9-9352-4589-82bd-3ea0f048181a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7qxms" Sep 08 02:23:17.513276 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:17.513225 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3deb3cf9-9352-4589-82bd-3ea0f048181a-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-7qxms\" (UID: \"3deb3cf9-9352-4589-82bd-3ea0f048181a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7qxms" Sep 08 02:23:17.515868 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:17.515847 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3deb3cf9-9352-4589-82bd-3ea0f048181a-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-7qxms\" (UID: \"3deb3cf9-9352-4589-82bd-3ea0f048181a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7qxms" Sep 08 02:23:17.515974 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:17.515939 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3deb3cf9-9352-4589-82bd-3ea0f048181a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-7qxms\" (UID: \"3deb3cf9-9352-4589-82bd-3ea0f048181a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7qxms" Sep 08 02:23:17.521114 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:17.521090 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74pk7\" (UniqueName: \"kubernetes.io/projected/3deb3cf9-9352-4589-82bd-3ea0f048181a-kube-api-access-74pk7\") pod \"prometheus-operator-6f4946db56-7qxms\" (UID: \"3deb3cf9-9352-4589-82bd-3ea0f048181a\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-7qxms" Sep 08 02:23:17.628188 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:17.628142 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-7qxms" Sep 08 02:23:17.912237 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:17.912115 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-7qxms"] Sep 08 02:23:17.914973 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:23:17.914944 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3deb3cf9_9352_4589_82bd_3ea0f048181a.slice/crio-3f62c39a5d396c4346a62c278a3d8bd398edb083a7d29ba36fc4ffb2839b6dee WatchSource:0}: Error finding container 3f62c39a5d396c4346a62c278a3d8bd398edb083a7d29ba36fc4ffb2839b6dee: Status 404 returned error can't find the container with id 3f62c39a5d396c4346a62c278a3d8bd398edb083a7d29ba36fc4ffb2839b6dee Sep 08 02:23:18.518742 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:18.518697 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-7qxms" event={"ID":"3deb3cf9-9352-4589-82bd-3ea0f048181a","Type":"ContainerStarted","Data":"3f62c39a5d396c4346a62c278a3d8bd398edb083a7d29ba36fc4ffb2839b6dee"} Sep 08 02:23:19.529223 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:19.528389 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-7qxms" event={"ID":"3deb3cf9-9352-4589-82bd-3ea0f048181a","Type":"ContainerStarted","Data":"db0f72d3b613ccccba67ec07f7131e20ef367cf4c30dabf5fdf8e1bcb184136e"} Sep 08 02:23:19.529223 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:19.528483 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-7qxms" event={"ID":"3deb3cf9-9352-4589-82bd-3ea0f048181a","Type":"ContainerStarted","Data":"112c97fff048d2ae757bd3c94b2e591e1dddccb7faaaa7bcafa57351621bfd1d"} Sep 08 02:23:19.549371 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:19.549308 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-7qxms" podStartSLOduration=1.209513947 podStartE2EDuration="2.549291394s" totalImagesPullingTime="1.339777447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:23:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:23:19.548079302 +0000 UTC m=+95.079724168" watchObservedRunningTime="2026-09-08 02:23:19.549291394 +0000 UTC m=+95.080936253" Sep 08 02:23:21.825053 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:21.824971 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-627dq"] Sep 08 02:23:21.830391 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:21.830353 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:21.833773 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:21.833677 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mjs9s\"" Sep 08 02:23:21.833917 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:21.833787 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 08 02:23:21.834411 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:21.834363 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 08 02:23:21.834721 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:21.834686 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 08 02:23:21.948807 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:21.948771 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d8c7193e-0ea8-47a5-9989-1a413144bb8a-root\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:21.948989 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:21.948816 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d8c7193e-0ea8-47a5-9989-1a413144bb8a-node-exporter-textfile\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:21.948989 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:21.948868 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d8c7193e-0ea8-47a5-9989-1a413144bb8a-node-exporter-tls\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:21.948989 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:21.948945 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8c7193e-0ea8-47a5-9989-1a413144bb8a-sys\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:21.948989 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:21.948983 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d8c7193e-0ea8-47a5-9989-1a413144bb8a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:21.949208 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:21.949021 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8c7193e-0ea8-47a5-9989-1a413144bb8a-metrics-client-ca\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:21.949208 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:21.949056 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mcsv\" (UniqueName: \"kubernetes.io/projected/d8c7193e-0ea8-47a5-9989-1a413144bb8a-kube-api-access-4mcsv\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:21.949208 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:21.949083 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d8c7193e-0ea8-47a5-9989-1a413144bb8a-node-exporter-wtmp\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:21.949208 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:21.949164 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d8c7193e-0ea8-47a5-9989-1a413144bb8a-node-exporter-accelerators-collector-config\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:22.050303 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.050243 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d8c7193e-0ea8-47a5-9989-1a413144bb8a-node-exporter-wtmp\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:22.050983 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.050327 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d8c7193e-0ea8-47a5-9989-1a413144bb8a-node-exporter-accelerators-collector-config\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:22.050983 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.050393 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d8c7193e-0ea8-47a5-9989-1a413144bb8a-root\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:22.050983 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.050417 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d8c7193e-0ea8-47a5-9989-1a413144bb8a-node-exporter-textfile\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:22.050983 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.050445 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d8c7193e-0ea8-47a5-9989-1a413144bb8a-node-exporter-tls\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:22.050983 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.050496 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8c7193e-0ea8-47a5-9989-1a413144bb8a-sys\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:22.050983 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.050522 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d8c7193e-0ea8-47a5-9989-1a413144bb8a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:22.050983 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.050550 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8c7193e-0ea8-47a5-9989-1a413144bb8a-metrics-client-ca\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:22.050983 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.050581 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mcsv\" (UniqueName: \"kubernetes.io/projected/d8c7193e-0ea8-47a5-9989-1a413144bb8a-kube-api-access-4mcsv\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:22.050983 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.050624 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d8c7193e-0ea8-47a5-9989-1a413144bb8a-root\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:22.050983 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.050641 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8c7193e-0ea8-47a5-9989-1a413144bb8a-sys\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:22.050983 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.050888 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d8c7193e-0ea8-47a5-9989-1a413144bb8a-node-exporter-wtmp\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:22.051689 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.051183 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8c7193e-0ea8-47a5-9989-1a413144bb8a-metrics-client-ca\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:22.051689 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.051323 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d8c7193e-0ea8-47a5-9989-1a413144bb8a-node-exporter-textfile\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:22.051689 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.051676 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d8c7193e-0ea8-47a5-9989-1a413144bb8a-node-exporter-accelerators-collector-config\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:22.053347 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.053319 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d8c7193e-0ea8-47a5-9989-1a413144bb8a-node-exporter-tls\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:22.053450 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.053397 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d8c7193e-0ea8-47a5-9989-1a413144bb8a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:22.060325 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.060305 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mcsv\" (UniqueName: \"kubernetes.io/projected/d8c7193e-0ea8-47a5-9989-1a413144bb8a-kube-api-access-4mcsv\") pod \"node-exporter-627dq\" (UID: \"d8c7193e-0ea8-47a5-9989-1a413144bb8a\") " pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:22.143094 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.143054 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-627dq" Sep 08 02:23:22.656476 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.656430 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls\") pod \"dns-default-qhkj2\" (UID: \"382809a0-5fcd-4635-8ab5-f7e83672b1a5\") " pod="openshift-dns/dns-default-qhkj2" Sep 08 02:23:22.659806 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.659766 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/382809a0-5fcd-4635-8ab5-f7e83672b1a5-metrics-tls\") pod \"dns-default-qhkj2\" (UID: \"382809a0-5fcd-4635-8ab5-f7e83672b1a5\") " pod="openshift-dns/dns-default-qhkj2" Sep 08 02:23:22.679765 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.679548 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-56wgd\"" Sep 08 02:23:22.687321 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.687254 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qhkj2" Sep 08 02:23:22.747950 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.747909 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 02:23:22.752382 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.752343 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.755915 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.755883 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 08 02:23:22.756082 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.755923 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 08 02:23:22.756082 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.755993 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 08 02:23:22.756302 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.756283 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 08 02:23:22.756377 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.756303 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 08 02:23:22.757309 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.757287 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 08 02:23:22.757488 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.757396 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-69stg\"" Sep 08 02:23:22.757764 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.757741 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 08 02:23:22.757960 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.757941 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 08 02:23:22.762858 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.762786 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 08 02:23:22.772053 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.772029 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 02:23:22.859056 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.858428 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cc563bc7-8836-4951-8ffa-3517967f4cab-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.859056 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.858492 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cc563bc7-8836-4951-8ffa-3517967f4cab-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.859056 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.858540 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cc563bc7-8836-4951-8ffa-3517967f4cab-config-out\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.859056 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.858565 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc563bc7-8836-4951-8ffa-3517967f4cab-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.859056 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.858595 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cc563bc7-8836-4951-8ffa-3517967f4cab-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.859056 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.858646 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cc563bc7-8836-4951-8ffa-3517967f4cab-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.859056 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.858667 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfwkk\" (UniqueName: \"kubernetes.io/projected/cc563bc7-8836-4951-8ffa-3517967f4cab-kube-api-access-rfwkk\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.859056 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.858694 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cc563bc7-8836-4951-8ffa-3517967f4cab-config-volume\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.859056 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.858720 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cc563bc7-8836-4951-8ffa-3517967f4cab-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.859056 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.858743 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cc563bc7-8836-4951-8ffa-3517967f4cab-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.859056 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.858791 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cc563bc7-8836-4951-8ffa-3517967f4cab-web-config\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.859056 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.858815 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc563bc7-8836-4951-8ffa-3517967f4cab-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.859056 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.858846 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cc563bc7-8836-4951-8ffa-3517967f4cab-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.959436 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.959354 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cc563bc7-8836-4951-8ffa-3517967f4cab-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.959436 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.959400 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cc563bc7-8836-4951-8ffa-3517967f4cab-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.959436 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.959431 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cc563bc7-8836-4951-8ffa-3517967f4cab-config-out\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.959738 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.959449 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc563bc7-8836-4951-8ffa-3517967f4cab-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.959738 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.959494 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cc563bc7-8836-4951-8ffa-3517967f4cab-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.959738 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.959547 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cc563bc7-8836-4951-8ffa-3517967f4cab-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.959738 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.959572 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfwkk\" (UniqueName: \"kubernetes.io/projected/cc563bc7-8836-4951-8ffa-3517967f4cab-kube-api-access-rfwkk\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.959738 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.959600 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cc563bc7-8836-4951-8ffa-3517967f4cab-config-volume\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.960025 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.959878 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cc563bc7-8836-4951-8ffa-3517967f4cab-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.960025 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.959920 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cc563bc7-8836-4951-8ffa-3517967f4cab-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.960025 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.959969 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cc563bc7-8836-4951-8ffa-3517967f4cab-web-config\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.960025 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.960001 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc563bc7-8836-4951-8ffa-3517967f4cab-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.960226 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.960039 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cc563bc7-8836-4951-8ffa-3517967f4cab-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.960226 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.960058 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cc563bc7-8836-4951-8ffa-3517967f4cab-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.961260 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.961237 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc563bc7-8836-4951-8ffa-3517967f4cab-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.961684 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.961645 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc563bc7-8836-4951-8ffa-3517967f4cab-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.963587 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.963546 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cc563bc7-8836-4951-8ffa-3517967f4cab-web-config\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.963956 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.963910 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cc563bc7-8836-4951-8ffa-3517967f4cab-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.963956 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.963937 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cc563bc7-8836-4951-8ffa-3517967f4cab-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.964187 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.964142 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cc563bc7-8836-4951-8ffa-3517967f4cab-config-out\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.964619 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.964596 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cc563bc7-8836-4951-8ffa-3517967f4cab-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.964619 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.964612 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cc563bc7-8836-4951-8ffa-3517967f4cab-config-volume\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.964814 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.964788 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cc563bc7-8836-4951-8ffa-3517967f4cab-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.965272 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.965227 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cc563bc7-8836-4951-8ffa-3517967f4cab-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.965374 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.965351 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cc563bc7-8836-4951-8ffa-3517967f4cab-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:22.968110 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:22.968092 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfwkk\" (UniqueName: \"kubernetes.io/projected/cc563bc7-8836-4951-8ffa-3517967f4cab-kube-api-access-rfwkk\") pod \"alertmanager-main-0\" (UID: \"cc563bc7-8836-4951-8ffa-3517967f4cab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:23.067957 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:23.067921 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 02:23:26.169006 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.168622 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-697fdf7c8b-7stmv"] Sep 08 02:23:26.195871 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.195838 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-697fdf7c8b-7stmv"] Sep 08 02:23:26.196020 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.195976 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:26.199255 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.199227 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 08 02:23:26.200446 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.200418 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 08 02:23:26.200551 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.200456 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 08 02:23:26.200551 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.200529 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-uvipevq2dbrb\"" Sep 08 02:23:26.200671 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.200527 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 08 02:23:26.200671 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.200587 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-qps2j\"" Sep 08 02:23:26.287742 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.287705 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bbe38cf3-5b49-4c35-b3cf-31e5b82172e8-audit-log\") pod \"metrics-server-697fdf7c8b-7stmv\" (UID: \"bbe38cf3-5b49-4c35-b3cf-31e5b82172e8\") " pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:26.287917 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.287762 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbe38cf3-5b49-4c35-b3cf-31e5b82172e8-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-697fdf7c8b-7stmv\" (UID: \"bbe38cf3-5b49-4c35-b3cf-31e5b82172e8\") " pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:26.287917 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.287848 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bbe38cf3-5b49-4c35-b3cf-31e5b82172e8-secret-metrics-server-tls\") pod \"metrics-server-697fdf7c8b-7stmv\" (UID: \"bbe38cf3-5b49-4c35-b3cf-31e5b82172e8\") " pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:26.288045 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.287939 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bbe38cf3-5b49-4c35-b3cf-31e5b82172e8-secret-metrics-server-client-certs\") pod \"metrics-server-697fdf7c8b-7stmv\" (UID: \"bbe38cf3-5b49-4c35-b3cf-31e5b82172e8\") " pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:26.288045 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.287979 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc97c\" (UniqueName: \"kubernetes.io/projected/bbe38cf3-5b49-4c35-b3cf-31e5b82172e8-kube-api-access-fc97c\") pod \"metrics-server-697fdf7c8b-7stmv\" (UID: \"bbe38cf3-5b49-4c35-b3cf-31e5b82172e8\") " pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:26.288045 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.288018 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bbe38cf3-5b49-4c35-b3cf-31e5b82172e8-client-ca-bundle\") pod \"metrics-server-697fdf7c8b-7stmv\" (UID: \"bbe38cf3-5b49-4c35-b3cf-31e5b82172e8\") " pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:26.288204 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.288046 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bbe38cf3-5b49-4c35-b3cf-31e5b82172e8-metrics-server-audit-profiles\") pod \"metrics-server-697fdf7c8b-7stmv\" (UID: \"bbe38cf3-5b49-4c35-b3cf-31e5b82172e8\") " pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:26.389356 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.389319 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bbe38cf3-5b49-4c35-b3cf-31e5b82172e8-secret-metrics-server-client-certs\") pod \"metrics-server-697fdf7c8b-7stmv\" (UID: \"bbe38cf3-5b49-4c35-b3cf-31e5b82172e8\") " pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:26.389547 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.389376 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc97c\" (UniqueName: \"kubernetes.io/projected/bbe38cf3-5b49-4c35-b3cf-31e5b82172e8-kube-api-access-fc97c\") pod \"metrics-server-697fdf7c8b-7stmv\" (UID: \"bbe38cf3-5b49-4c35-b3cf-31e5b82172e8\") " pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:26.389547 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.389408 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bbe38cf3-5b49-4c35-b3cf-31e5b82172e8-client-ca-bundle\") pod \"metrics-server-697fdf7c8b-7stmv\" (UID: \"bbe38cf3-5b49-4c35-b3cf-31e5b82172e8\") " pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:26.389669 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.389580 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bbe38cf3-5b49-4c35-b3cf-31e5b82172e8-metrics-server-audit-profiles\") pod \"metrics-server-697fdf7c8b-7stmv\" (UID: \"bbe38cf3-5b49-4c35-b3cf-31e5b82172e8\") " pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:26.389724 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.389676 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bbe38cf3-5b49-4c35-b3cf-31e5b82172e8-audit-log\") pod \"metrics-server-697fdf7c8b-7stmv\" (UID: \"bbe38cf3-5b49-4c35-b3cf-31e5b82172e8\") " pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:26.389724 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.389718 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbe38cf3-5b49-4c35-b3cf-31e5b82172e8-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-697fdf7c8b-7stmv\" (UID: \"bbe38cf3-5b49-4c35-b3cf-31e5b82172e8\") " pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:26.389861 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.389834 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bbe38cf3-5b49-4c35-b3cf-31e5b82172e8-secret-metrics-server-tls\") pod \"metrics-server-697fdf7c8b-7stmv\" (UID: \"bbe38cf3-5b49-4c35-b3cf-31e5b82172e8\") " pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:26.390057 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.390034 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bbe38cf3-5b49-4c35-b3cf-31e5b82172e8-audit-log\") pod \"metrics-server-697fdf7c8b-7stmv\" (UID: \"bbe38cf3-5b49-4c35-b3cf-31e5b82172e8\") " pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:26.390496 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.390451 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbe38cf3-5b49-4c35-b3cf-31e5b82172e8-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-697fdf7c8b-7stmv\" (UID: \"bbe38cf3-5b49-4c35-b3cf-31e5b82172e8\") " pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:26.390635 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.390616 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bbe38cf3-5b49-4c35-b3cf-31e5b82172e8-metrics-server-audit-profiles\") pod \"metrics-server-697fdf7c8b-7stmv\" (UID: \"bbe38cf3-5b49-4c35-b3cf-31e5b82172e8\") " pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:26.392262 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.392221 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bbe38cf3-5b49-4c35-b3cf-31e5b82172e8-client-ca-bundle\") pod \"metrics-server-697fdf7c8b-7stmv\" (UID: \"bbe38cf3-5b49-4c35-b3cf-31e5b82172e8\") " pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:26.392829 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.392785 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bbe38cf3-5b49-4c35-b3cf-31e5b82172e8-secret-metrics-server-tls\") pod \"metrics-server-697fdf7c8b-7stmv\" (UID: \"bbe38cf3-5b49-4c35-b3cf-31e5b82172e8\") " pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:26.393368 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.393348 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bbe38cf3-5b49-4c35-b3cf-31e5b82172e8-secret-metrics-server-client-certs\") pod \"metrics-server-697fdf7c8b-7stmv\" (UID: \"bbe38cf3-5b49-4c35-b3cf-31e5b82172e8\") " pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:26.410576 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.410067 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc97c\" (UniqueName: \"kubernetes.io/projected/bbe38cf3-5b49-4c35-b3cf-31e5b82172e8-kube-api-access-fc97c\") pod \"metrics-server-697fdf7c8b-7stmv\" (UID: \"bbe38cf3-5b49-4c35-b3cf-31e5b82172e8\") " pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:26.507247 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:26.507154 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:27.083909 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.083528 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n"] Sep 08 02:23:27.097784 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.097762 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.101287 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.101266 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-wjdhs\"" Sep 08 02:23:27.101386 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.101266 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 08 02:23:27.102553 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.102533 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 08 02:23:27.102827 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.102804 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 08 02:23:27.103159 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.103143 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 08 02:23:27.106167 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.105795 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 08 02:23:27.107272 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.107253 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 08 02:23:27.117373 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.117352 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n"] Sep 08 02:23:27.198943 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.198909 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38ac877f-56e9-418b-9fa0-0456fbc73c7e-serving-certs-ca-bundle\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.198943 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.198949 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38ac877f-56e9-418b-9fa0-0456fbc73c7e-metrics-client-ca\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.199327 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.198987 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/38ac877f-56e9-418b-9fa0-0456fbc73c7e-federate-client-tls\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.199327 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.199041 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/38ac877f-56e9-418b-9fa0-0456fbc73c7e-secret-telemeter-client\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.199327 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.199063 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b9cr\" (UniqueName: \"kubernetes.io/projected/38ac877f-56e9-418b-9fa0-0456fbc73c7e-kube-api-access-4b9cr\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.199327 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.199090 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/38ac877f-56e9-418b-9fa0-0456fbc73c7e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.199327 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.199134 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/38ac877f-56e9-418b-9fa0-0456fbc73c7e-telemeter-client-tls\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.199327 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.199185 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38ac877f-56e9-418b-9fa0-0456fbc73c7e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.283514 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.283486 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qhkj2"] Sep 08 02:23:27.284667 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.284646 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-697fdf7c8b-7stmv"] Sep 08 02:23:27.298480 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.298436 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 02:23:27.300228 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.300196 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/38ac877f-56e9-418b-9fa0-0456fbc73c7e-telemeter-client-tls\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.300274 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.300263 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38ac877f-56e9-418b-9fa0-0456fbc73c7e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.300307 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.300299 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38ac877f-56e9-418b-9fa0-0456fbc73c7e-serving-certs-ca-bundle\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.300353 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.300323 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38ac877f-56e9-418b-9fa0-0456fbc73c7e-metrics-client-ca\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.300399 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.300355 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/38ac877f-56e9-418b-9fa0-0456fbc73c7e-federate-client-tls\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.300440 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.300401 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/38ac877f-56e9-418b-9fa0-0456fbc73c7e-secret-telemeter-client\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.300440 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.300426 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b9cr\" (UniqueName: \"kubernetes.io/projected/38ac877f-56e9-418b-9fa0-0456fbc73c7e-kube-api-access-4b9cr\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.300560 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.300455 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/38ac877f-56e9-418b-9fa0-0456fbc73c7e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.301134 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.301099 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38ac877f-56e9-418b-9fa0-0456fbc73c7e-metrics-client-ca\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.301326 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.301298 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38ac877f-56e9-418b-9fa0-0456fbc73c7e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.301441 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.301367 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38ac877f-56e9-418b-9fa0-0456fbc73c7e-serving-certs-ca-bundle\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.302832 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.302814 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/38ac877f-56e9-418b-9fa0-0456fbc73c7e-federate-client-tls\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.302940 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.302920 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/38ac877f-56e9-418b-9fa0-0456fbc73c7e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.303157 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.303137 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/38ac877f-56e9-418b-9fa0-0456fbc73c7e-secret-telemeter-client\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.303157 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.303150 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/38ac877f-56e9-418b-9fa0-0456fbc73c7e-telemeter-client-tls\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.309488 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.309442 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b9cr\" (UniqueName: \"kubernetes.io/projected/38ac877f-56e9-418b-9fa0-0456fbc73c7e-kube-api-access-4b9cr\") pod \"telemeter-client-6589c8c8fc-gjf6n\" (UID: \"38ac877f-56e9-418b-9fa0-0456fbc73c7e\") " pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.352968 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:23:27.352924 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbe38cf3_5b49_4c35_b3cf_31e5b82172e8.slice/crio-dd67e585b7292d2b556fd5a66fd81f62df26e8f4c482718b62de0e8efa4f50ae WatchSource:0}: Error finding container dd67e585b7292d2b556fd5a66fd81f62df26e8f4c482718b62de0e8efa4f50ae: Status 404 returned error can't find the container with id dd67e585b7292d2b556fd5a66fd81f62df26e8f4c482718b62de0e8efa4f50ae Sep 08 02:23:27.353755 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:23:27.353729 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod382809a0_5fcd_4635_8ab5_f7e83672b1a5.slice/crio-5d7c29a51313c68e8c09d924aa293d7fd8fd575ffb0dff140588da97ec8a572f WatchSource:0}: Error finding container 5d7c29a51313c68e8c09d924aa293d7fd8fd575ffb0dff140588da97ec8a572f: Status 404 returned error can't find the container with id 5d7c29a51313c68e8c09d924aa293d7fd8fd575ffb0dff140588da97ec8a572f Sep 08 02:23:27.354520 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:23:27.354492 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc563bc7_8836_4951_8ffa_3517967f4cab.slice/crio-89a3f040c5d1f295361ea0511c08698592d2bf8e4bd367100b45647a7c4c4eb4 WatchSource:0}: Error finding container 89a3f040c5d1f295361ea0511c08698592d2bf8e4bd367100b45647a7c4c4eb4: Status 404 returned error can't find the container with id 89a3f040c5d1f295361ea0511c08698592d2bf8e4bd367100b45647a7c4c4eb4 Sep 08 02:23:27.409509 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.409472 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" Sep 08 02:23:27.434534 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.434513 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-c8fz9" Sep 08 02:23:27.563347 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.563287 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-gc2gg" event={"ID":"2d721641-4617-481e-8947-e5058100df47","Type":"ContainerStarted","Data":"72cdf87c1dd75d75b485368b816067ce1b2d717861b556374cf28b60dba3434f"} Sep 08 02:23:27.564910 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.564872 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" event={"ID":"bbe38cf3-5b49-4c35-b3cf-31e5b82172e8","Type":"ContainerStarted","Data":"dd67e585b7292d2b556fd5a66fd81f62df26e8f4c482718b62de0e8efa4f50ae"} Sep 08 02:23:27.566379 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.566346 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-627dq" event={"ID":"d8c7193e-0ea8-47a5-9989-1a413144bb8a","Type":"ContainerStarted","Data":"4c7d34da6cb74fb7e8fb0a4f6dfe9cc4bccc61fa153ac10df1376eb53bc78ce7"} Sep 08 02:23:27.567791 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.567767 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qhkj2" event={"ID":"382809a0-5fcd-4635-8ab5-f7e83672b1a5","Type":"ContainerStarted","Data":"5d7c29a51313c68e8c09d924aa293d7fd8fd575ffb0dff140588da97ec8a572f"} Sep 08 02:23:27.569147 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.569123 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc563bc7-8836-4951-8ffa-3517967f4cab","Type":"ContainerStarted","Data":"89a3f040c5d1f295361ea0511c08698592d2bf8e4bd367100b45647a7c4c4eb4"} Sep 08 02:23:27.719747 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:27.719711 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n"] Sep 08 02:23:27.723456 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:23:27.723424 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38ac877f_56e9_418b_9fa0_0456fbc73c7e.slice/crio-5a006f5a69f20a599061d3d8873315db85bd411813e44f1e6ac5f9e79455e22b WatchSource:0}: Error finding container 5a006f5a69f20a599061d3d8873315db85bd411813e44f1e6ac5f9e79455e22b: Status 404 returned error can't find the container with id 5a006f5a69f20a599061d3d8873315db85bd411813e44f1e6ac5f9e79455e22b Sep 08 02:23:28.080366 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.080328 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 02:23:28.123075 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.123041 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 02:23:28.123355 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.123334 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.132151 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.129600 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 08 02:23:28.132151 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.129981 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 08 02:23:28.132151 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.130171 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 08 02:23:28.132151 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.130433 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 08 02:23:28.132151 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.130995 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-bkpbfoprqn0h1\"" Sep 08 02:23:28.132151 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.131277 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qg68n\"" Sep 08 02:23:28.132151 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.131584 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 08 02:23:28.132151 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.131862 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 08 02:23:28.134938 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.134077 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 08 02:23:28.134938 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.134319 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 08 02:23:28.134938 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.134518 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 08 02:23:28.134938 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.134751 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 08 02:23:28.138747 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.135117 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 08 02:23:28.138747 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.136939 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 08 02:23:28.160625 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.156633 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 08 02:23:28.160625 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.157051 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 08 02:23:28.212161 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.212114 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.212771 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.212692 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.212921 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.212899 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.213024 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.212950 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.213024 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.212973 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.213024 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.213009 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.213175 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.213062 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.213175 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.213112 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.213175 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.213142 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.213316 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.213175 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.213316 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.213201 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.213316 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.213230 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.213316 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.213256 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-config\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.213316 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.213280 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.213316 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.213302 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.213656 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.213325 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.213656 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.213351 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.213656 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.213389 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-web-config\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.213656 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.213415 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5wll\" (UniqueName: \"kubernetes.io/projected/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-kube-api-access-m5wll\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.213656 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.213453 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-config-out\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.320329 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.314726 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.320329 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.314785 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.320329 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.314817 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.320329 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.314858 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.320329 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.314881 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.320329 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.314916 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.320329 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.314958 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.320329 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.314999 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.320329 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.315026 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.320329 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.315054 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.320329 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.315080 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.320329 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.315108 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.320329 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.315134 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-config\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.320329 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.315157 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.320329 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.315180 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.320329 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.315199 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.320329 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.315223 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.321296 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.315259 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-web-config\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.321296 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.315282 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5wll\" (UniqueName: \"kubernetes.io/projected/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-kube-api-access-m5wll\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.321296 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.315316 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-config-out\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.321296 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.315796 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.321296 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.315828 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.321296 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.316066 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.321296 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.316356 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.321296 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.316437 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.321296 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.317298 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.321296 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.319054 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-config\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.321296 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.321168 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.321871 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.321484 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.321934 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.321884 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.323665 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.323193 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.324233 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.324180 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.325365 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.325321 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.326308 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.326199 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-config-out\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.326308 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.326257 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.326776 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.326736 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.326951 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.326906 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.328892 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.328868 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-web-config\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.328978 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.328946 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.336929 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.336844 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5wll\" (UniqueName: \"kubernetes.io/projected/6c2dd45b-924d-40a5-81c0-ed2f72d63ab0-kube-api-access-m5wll\") pod \"prometheus-k8s-0\" (UID: \"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.461873 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.461832 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:28.576447 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.575798 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" event={"ID":"38ac877f-56e9-418b-9fa0-0456fbc73c7e","Type":"ContainerStarted","Data":"5a006f5a69f20a599061d3d8873315db85bd411813e44f1e6ac5f9e79455e22b"} Sep 08 02:23:28.576447 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.576348 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-gc2gg" Sep 08 02:23:28.592664 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.592556 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-gc2gg" Sep 08 02:23:28.605401 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:28.605276 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-gc2gg" podStartSLOduration=2.413937732 podStartE2EDuration="19.605260852s" totalImagesPullingTime="17.19132312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:23:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:23:28.603233786 +0000 UTC m=+104.134878653" watchObservedRunningTime="2026-09-08 02:23:28.605260852 +0000 UTC m=+104.136905717" Sep 08 02:23:29.553584 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:29.553502 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 08 02:23:29.587450 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:29.587412 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0","Type":"ContainerStarted","Data":"ec32dd60a9a9b6938d21be52538811fb8143ed56b1cff8a9c2e22a6b812ae66c"} Sep 08 02:23:29.591276 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:29.591242 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-627dq" event={"ID":"d8c7193e-0ea8-47a5-9989-1a413144bb8a","Type":"ContainerStarted","Data":"5cf18553a02e99a0be995cc514bc1f513d18e9403f058b01cb208ce48b44f334"} Sep 08 02:23:30.598338 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:30.598299 2560 generic.go:356] "Generic (PLEG): container finished" podID="6c2dd45b-924d-40a5-81c0-ed2f72d63ab0" containerID="4a86ec1ac0b578def9396f7674b53ab0f4b4f5a22681383b144aa5f0ef980896" exitCode=0 Sep 08 02:23:30.598917 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:30.598396 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0","Type":"ContainerDied","Data":"4a86ec1ac0b578def9396f7674b53ab0f4b4f5a22681383b144aa5f0ef980896"} Sep 08 02:23:30.602215 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:30.602116 2560 generic.go:356] "Generic (PLEG): container finished" podID="d8c7193e-0ea8-47a5-9989-1a413144bb8a" containerID="5cf18553a02e99a0be995cc514bc1f513d18e9403f058b01cb208ce48b44f334" exitCode=0 Sep 08 02:23:30.602408 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:30.602276 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-627dq" event={"ID":"d8c7193e-0ea8-47a5-9989-1a413144bb8a","Type":"ContainerDied","Data":"5cf18553a02e99a0be995cc514bc1f513d18e9403f058b01cb208ce48b44f334"} Sep 08 02:23:30.602408 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:30.602323 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-627dq" event={"ID":"d8c7193e-0ea8-47a5-9989-1a413144bb8a","Type":"ContainerStarted","Data":"f08a92d4f3b61d290776e7616f997d4c9e068723f700a320f339d475ddee33cd"} Sep 08 02:23:30.602790 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:30.602764 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-627dq" event={"ID":"d8c7193e-0ea8-47a5-9989-1a413144bb8a","Type":"ContainerStarted","Data":"44b8480442a0edf7fec521744c8ab6d0034dcb704496c39e7863825910c3600e"} Sep 08 02:23:30.604992 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:30.604968 2560 generic.go:356] "Generic (PLEG): container finished" podID="cc563bc7-8836-4951-8ffa-3517967f4cab" containerID="0e9a8e86118785abf8e6dce8b72002668363c4e54064b2a95f63d7c2b13f2e6f" exitCode=0 Sep 08 02:23:30.605150 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:30.605111 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc563bc7-8836-4951-8ffa-3517967f4cab","Type":"ContainerDied","Data":"0e9a8e86118785abf8e6dce8b72002668363c4e54064b2a95f63d7c2b13f2e6f"} Sep 08 02:23:30.690568 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:30.690198 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-627dq" podStartSLOduration=7.366562493 podStartE2EDuration="9.690177409s" totalImagesPullingTime="2.217297341s" totalInitContainerRuntime="106.317575ms" isStatefulPod=true podCreationTimestamp="2026-09-08 02:23:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:23:30.688616223 +0000 UTC m=+106.220261088" watchObservedRunningTime="2026-09-08 02:23:30.690177409 +0000 UTC m=+106.221822275" Sep 08 02:23:32.627721 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:32.627602 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" event={"ID":"bbe38cf3-5b49-4c35-b3cf-31e5b82172e8","Type":"ContainerStarted","Data":"4759884eeb46c03122ff5e043c4866343fb40e3ad5496c27ad3b4e040b5e054d"} Sep 08 02:23:33.652059 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:33.650071 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" event={"ID":"38ac877f-56e9-418b-9fa0-0456fbc73c7e","Type":"ContainerStarted","Data":"665ba104b3fb75e8ccd415ec45a8b57169ca9e42b6ca66d994e4c4460b5dd4f7"} Sep 08 02:23:33.652059 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:33.650189 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" event={"ID":"38ac877f-56e9-418b-9fa0-0456fbc73c7e","Type":"ContainerStarted","Data":"a63c8e6f6e388f4aa165a9d4dfa2600c79714a466ed6558054522295f8469971"} Sep 08 02:23:33.652059 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:33.650221 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" event={"ID":"38ac877f-56e9-418b-9fa0-0456fbc73c7e","Type":"ContainerStarted","Data":"bfccf66f0b75e0c199ce490ae83e09afda3e5e4e4213436e6517b484569d0be7"} Sep 08 02:23:33.656427 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:33.656115 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qhkj2" event={"ID":"382809a0-5fcd-4635-8ab5-f7e83672b1a5","Type":"ContainerStarted","Data":"7fd2a34631cbc396657d7c60599a9ad8e6b5d36c6eb9e172840711621fcfb1c9"} Sep 08 02:23:33.656427 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:33.656166 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qhkj2" event={"ID":"382809a0-5fcd-4635-8ab5-f7e83672b1a5","Type":"ContainerStarted","Data":"83255aee7c6f3286fa7f72d54edb4676db07cc64a3fedec36dbd10282f05b9eb"} Sep 08 02:23:33.656427 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:33.656222 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qhkj2" Sep 08 02:23:33.679589 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:33.679016 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6589c8c8fc-gjf6n" podStartSLOduration=1.955262676 podStartE2EDuration="6.678995385s" totalImagesPullingTime="4.723732709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:23:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:23:33.675704356 +0000 UTC m=+109.207349426" watchObservedRunningTime="2026-09-08 02:23:33.678995385 +0000 UTC m=+109.210640251" Sep 08 02:23:33.732261 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:33.732203 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qhkj2" podStartSLOduration=70.659226879 podStartE2EDuration="1m15.732182881s" totalImagesPullingTime="5.072956002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:22:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:23:33.698244649 +0000 UTC m=+109.229889515" watchObservedRunningTime="2026-09-08 02:23:33.732182881 +0000 UTC m=+109.263827747" Sep 08 02:23:33.733857 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:33.733812 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" podStartSLOduration=2.667819231 podStartE2EDuration="7.733799132s" totalImagesPullingTime="5.065979901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:23:33.730246381 +0000 UTC m=+109.261891248" watchObservedRunningTime="2026-09-08 02:23:33.733799132 +0000 UTC m=+109.265443998" Sep 08 02:23:34.667143 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:34.667049 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc563bc7-8836-4951-8ffa-3517967f4cab","Type":"ContainerStarted","Data":"233aa081ab3fd80b96252d36d989c9b755662c042722138949898a5fd8386d26"} Sep 08 02:23:35.675953 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:35.675914 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc563bc7-8836-4951-8ffa-3517967f4cab","Type":"ContainerStarted","Data":"983e89da81f5649ecb8785b2c65adb789234e1f517cc3448115aaf2a5f6173dd"} Sep 08 02:23:35.676523 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:35.676004 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc563bc7-8836-4951-8ffa-3517967f4cab","Type":"ContainerStarted","Data":"4fdc85ba919304fa27d1443c7bfdab218c1a5204b1bd0d73d4bddff71e5e9c16"} Sep 08 02:23:36.683523 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:36.683494 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc563bc7-8836-4951-8ffa-3517967f4cab","Type":"ContainerStarted","Data":"7e1cafca4d30fb4ae2d6297ffc601b1dfac1853465ced4601d1184b77985e257"} Sep 08 02:23:36.683897 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:36.683615 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc563bc7-8836-4951-8ffa-3517967f4cab","Type":"ContainerStarted","Data":"1b1c4b3bfbc885a9b2ce3f317884d1a62780fc7b1593110fdaa6976300d60e78"} Sep 08 02:23:36.685400 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:36.685379 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0","Type":"ContainerStarted","Data":"f6b4267bdae12c3e77bbed726792a1708540b171315896d9bc53a28b5a8df4d1"} Sep 08 02:23:37.693786 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:37.693745 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0","Type":"ContainerStarted","Data":"07f2c748f54da593aa3abf54e98203d9272c3eedb81148dd8fdc914bb539a019"} Sep 08 02:23:38.701161 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:38.701125 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc563bc7-8836-4951-8ffa-3517967f4cab","Type":"ContainerStarted","Data":"b8dcf039964c7375b9976bdd0a3445f7acb3a63c2174ba8ad52835727409262e"} Sep 08 02:23:38.733255 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:38.733198 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=9.167958673 podStartE2EDuration="16.733182554s" totalImagesPullingTime="7.546796067s" totalInitContainerRuntime="18.427814ms" isStatefulPod=false podCreationTimestamp="2026-09-08 02:23:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:23:38.732709249 +0000 UTC m=+114.264354113" watchObservedRunningTime="2026-09-08 02:23:38.733182554 +0000 UTC m=+114.264827419" Sep 08 02:23:39.708146 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:39.708107 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0","Type":"ContainerStarted","Data":"b75ef7837adf50c9c13810371c2f88f2f71429c3107f164408f206aec34a6a5f"} Sep 08 02:23:39.708612 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:39.708218 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0","Type":"ContainerStarted","Data":"bef8488e0d05123b0249a1706322f8ea95152648278a09c3162fa2668a6c545d"} Sep 08 02:23:39.708612 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:39.708267 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0","Type":"ContainerStarted","Data":"603b7bce8b7e4c18a4ee13d169a0e2458f621aab1bdb95da5e8133e05be6091d"} Sep 08 02:23:39.708612 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:39.708310 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6c2dd45b-924d-40a5-81c0-ed2f72d63ab0","Type":"ContainerStarted","Data":"eb3a9ba9bff01a1adb103a14dc6e09f5eaf8383e87412d9236c29e7fcc6fc04c"} Sep 08 02:23:39.741037 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:39.740971 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.464484203 podStartE2EDuration="11.740950508s" totalImagesPullingTime="8.217604285s" totalInitContainerRuntime="58.86202ms" isStatefulPod=false podCreationTimestamp="2026-09-08 02:23:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:23:39.738630253 +0000 UTC m=+115.270275118" watchObservedRunningTime="2026-09-08 02:23:39.740950508 +0000 UTC m=+115.272595372" Sep 08 02:23:43.462159 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:43.462107 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:23:43.684728 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:43.684693 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qhkj2" Sep 08 02:23:46.507499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:46.507442 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:46.507929 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:46.507545 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:23:54.882434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:54.882381 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs\") pod \"network-metrics-daemon-mt4gl\" (UID: \"ec8f0aba-d0c2-4d45-b25d-84e2790d49ec\") " pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:23:54.884706 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:54.884685 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec8f0aba-d0c2-4d45-b25d-84e2790d49ec-metrics-certs\") pod \"network-metrics-daemon-mt4gl\" (UID: \"ec8f0aba-d0c2-4d45-b25d-84e2790d49ec\") " pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:23:55.039811 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:55.039782 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-42xzn\"" Sep 08 02:23:55.047262 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:55.047242 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mt4gl" Sep 08 02:23:55.318670 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:55.318641 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mt4gl"] Sep 08 02:23:55.321137 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:23:55.321112 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec8f0aba_d0c2_4d45_b25d_84e2790d49ec.slice/crio-78a0d8924955d0b7eab91c83fa8dce2f2e9ae03d2ed912b87edf18b0df5d1ff1 WatchSource:0}: Error finding container 78a0d8924955d0b7eab91c83fa8dce2f2e9ae03d2ed912b87edf18b0df5d1ff1: Status 404 returned error can't find the container with id 78a0d8924955d0b7eab91c83fa8dce2f2e9ae03d2ed912b87edf18b0df5d1ff1 Sep 08 02:23:55.772808 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:55.772753 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mt4gl" event={"ID":"ec8f0aba-d0c2-4d45-b25d-84e2790d49ec","Type":"ContainerStarted","Data":"78a0d8924955d0b7eab91c83fa8dce2f2e9ae03d2ed912b87edf18b0df5d1ff1"} Sep 08 02:23:56.782728 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:56.782694 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mt4gl" event={"ID":"ec8f0aba-d0c2-4d45-b25d-84e2790d49ec","Type":"ContainerStarted","Data":"a07e0544f8629b03c9d02644f6457630d76c0b0379ed1deb95c34be95728047c"} Sep 08 02:23:56.783079 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:56.782773 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mt4gl" event={"ID":"ec8f0aba-d0c2-4d45-b25d-84e2790d49ec","Type":"ContainerStarted","Data":"98f68c5d9109d7e2d834e6b71054a41409cccde60e764eafc88057d912ed5668"} Sep 08 02:23:56.801669 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:23:56.801616 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mt4gl" podStartSLOduration=130.992632343 podStartE2EDuration="2m11.801602232s" totalImagesPullingTime="808.969889ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:23:56.800881373 +0000 UTC m=+132.332526240" watchObservedRunningTime="2026-09-08 02:23:56.801602232 +0000 UTC m=+132.333247097" Sep 08 02:24:03.813367 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:24:03.813330 2560 generic.go:356] "Generic (PLEG): container finished" podID="312a0520-e0c1-45b6-a9ac-e9f0b54df733" containerID="bcd5af9c0dba13629008983e08bacc0dbfa3461f7f4670fb0202f61904364c11" exitCode=0 Sep 08 02:24:03.813810 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:24:03.813407 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" event={"ID":"312a0520-e0c1-45b6-a9ac-e9f0b54df733","Type":"ContainerDied","Data":"bcd5af9c0dba13629008983e08bacc0dbfa3461f7f4670fb0202f61904364c11"} Sep 08 02:24:03.813810 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:24:03.813799 2560 scope.go:117] "RemoveContainer" containerID="bcd5af9c0dba13629008983e08bacc0dbfa3461f7f4670fb0202f61904364c11" Sep 08 02:24:04.818830 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:24:04.818795 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-2hccm" event={"ID":"312a0520-e0c1-45b6-a9ac-e9f0b54df733","Type":"ContainerStarted","Data":"0599d9d6acf6c53418e20df257e4693fc5035b54d51f3a44eaa7a36e51ebf4b7"} Sep 08 02:24:06.512663 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:24:06.512632 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:24:06.516482 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:24:06.516437 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-697fdf7c8b-7stmv" Sep 08 02:24:13.854335 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:24:13.854301 2560 generic.go:356] "Generic (PLEG): container finished" podID="61eceefd-089d-4d1f-9088-1793fd4fe32d" containerID="c21792e81e8d3d3c673f451273c9638c443535a0f609f0f200da8ecb2a44fa10" exitCode=0 Sep 08 02:24:13.854771 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:24:13.854349 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-drqqr" event={"ID":"61eceefd-089d-4d1f-9088-1793fd4fe32d","Type":"ContainerDied","Data":"c21792e81e8d3d3c673f451273c9638c443535a0f609f0f200da8ecb2a44fa10"} Sep 08 02:24:13.854771 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:24:13.854701 2560 scope.go:117] "RemoveContainer" containerID="c21792e81e8d3d3c673f451273c9638c443535a0f609f0f200da8ecb2a44fa10" Sep 08 02:24:14.861431 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:24:14.861391 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-drqqr" event={"ID":"61eceefd-089d-4d1f-9088-1793fd4fe32d","Type":"ContainerStarted","Data":"b7d91b02d889cc9469ba0c93f594baf0c4dab0fd11c5c953caef90a83ee723d7"} Sep 08 02:24:28.462222 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:24:28.462169 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:24:28.477899 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:24:28.477873 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:24:28.937243 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:24:28.937215 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 08 02:26:35.316732 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.316694 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2"] Sep 08 02:26:35.319810 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.319791 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" Sep 08 02:26:35.323545 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.323520 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 08 02:26:35.323659 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.323545 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 08 02:26:35.323828 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.323814 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\"" Sep 08 02:26:35.323913 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.323899 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\"" Sep 08 02:26:35.323960 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.323907 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\"" Sep 08 02:26:35.324650 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.324636 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-58nrg\"" Sep 08 02:26:35.339701 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.339678 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2"] Sep 08 02:26:35.411519 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.411486 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2414c1ef-e298-49fd-a86a-557585b4b622-manifests\") pod \"opendatahub-operator-controller-manager-5c89fb8b78-qjpw2\" (UID: \"2414c1ef-e298-49fd-a86a-557585b4b622\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" Sep 08 02:26:35.411656 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.411525 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2414c1ef-e298-49fd-a86a-557585b4b622-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c89fb8b78-qjpw2\" (UID: \"2414c1ef-e298-49fd-a86a-557585b4b622\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" Sep 08 02:26:35.411656 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.411570 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2414c1ef-e298-49fd-a86a-557585b4b622-tmp\") pod \"opendatahub-operator-controller-manager-5c89fb8b78-qjpw2\" (UID: \"2414c1ef-e298-49fd-a86a-557585b4b622\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" Sep 08 02:26:35.411656 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.411592 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2414c1ef-e298-49fd-a86a-557585b4b622-metrics-cert\") pod \"opendatahub-operator-controller-manager-5c89fb8b78-qjpw2\" (UID: \"2414c1ef-e298-49fd-a86a-557585b4b622\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" Sep 08 02:26:35.411656 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.411651 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc5pz\" (UniqueName: \"kubernetes.io/projected/2414c1ef-e298-49fd-a86a-557585b4b622-kube-api-access-sc5pz\") pod \"opendatahub-operator-controller-manager-5c89fb8b78-qjpw2\" (UID: \"2414c1ef-e298-49fd-a86a-557585b4b622\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" Sep 08 02:26:35.411788 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.411686 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2414c1ef-e298-49fd-a86a-557585b4b622-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c89fb8b78-qjpw2\" (UID: \"2414c1ef-e298-49fd-a86a-557585b4b622\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" Sep 08 02:26:35.512873 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.512848 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2414c1ef-e298-49fd-a86a-557585b4b622-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c89fb8b78-qjpw2\" (UID: \"2414c1ef-e298-49fd-a86a-557585b4b622\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" Sep 08 02:26:35.512989 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.512886 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2414c1ef-e298-49fd-a86a-557585b4b622-manifests\") pod \"opendatahub-operator-controller-manager-5c89fb8b78-qjpw2\" (UID: \"2414c1ef-e298-49fd-a86a-557585b4b622\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" Sep 08 02:26:35.512989 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.512904 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2414c1ef-e298-49fd-a86a-557585b4b622-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c89fb8b78-qjpw2\" (UID: \"2414c1ef-e298-49fd-a86a-557585b4b622\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" Sep 08 02:26:35.512989 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.512941 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2414c1ef-e298-49fd-a86a-557585b4b622-tmp\") pod \"opendatahub-operator-controller-manager-5c89fb8b78-qjpw2\" (UID: \"2414c1ef-e298-49fd-a86a-557585b4b622\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" Sep 08 02:26:35.512989 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.512975 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2414c1ef-e298-49fd-a86a-557585b4b622-metrics-cert\") pod \"opendatahub-operator-controller-manager-5c89fb8b78-qjpw2\" (UID: \"2414c1ef-e298-49fd-a86a-557585b4b622\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" Sep 08 02:26:35.513176 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.513033 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc5pz\" (UniqueName: \"kubernetes.io/projected/2414c1ef-e298-49fd-a86a-557585b4b622-kube-api-access-sc5pz\") pod \"opendatahub-operator-controller-manager-5c89fb8b78-qjpw2\" (UID: \"2414c1ef-e298-49fd-a86a-557585b4b622\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" Sep 08 02:26:35.513351 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.513321 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2414c1ef-e298-49fd-a86a-557585b4b622-manifests\") pod \"opendatahub-operator-controller-manager-5c89fb8b78-qjpw2\" (UID: \"2414c1ef-e298-49fd-a86a-557585b4b622\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" Sep 08 02:26:35.518484 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.516263 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2414c1ef-e298-49fd-a86a-557585b4b622-metrics-cert\") pod \"opendatahub-operator-controller-manager-5c89fb8b78-qjpw2\" (UID: \"2414c1ef-e298-49fd-a86a-557585b4b622\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" Sep 08 02:26:35.518484 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.516425 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2414c1ef-e298-49fd-a86a-557585b4b622-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c89fb8b78-qjpw2\" (UID: \"2414c1ef-e298-49fd-a86a-557585b4b622\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" Sep 08 02:26:35.518484 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.516428 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2414c1ef-e298-49fd-a86a-557585b4b622-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c89fb8b78-qjpw2\" (UID: \"2414c1ef-e298-49fd-a86a-557585b4b622\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" Sep 08 02:26:35.520887 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.520863 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2414c1ef-e298-49fd-a86a-557585b4b622-tmp\") pod \"opendatahub-operator-controller-manager-5c89fb8b78-qjpw2\" (UID: \"2414c1ef-e298-49fd-a86a-557585b4b622\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" Sep 08 02:26:35.529712 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.529689 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc5pz\" (UniqueName: \"kubernetes.io/projected/2414c1ef-e298-49fd-a86a-557585b4b622-kube-api-access-sc5pz\") pod \"opendatahub-operator-controller-manager-5c89fb8b78-qjpw2\" (UID: \"2414c1ef-e298-49fd-a86a-557585b4b622\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" Sep 08 02:26:35.629385 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.629356 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" Sep 08 02:26:35.911054 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:35.911028 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2"] Sep 08 02:26:35.913005 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:26:35.912974 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2414c1ef_e298_49fd_a86a_557585b4b622.slice/crio-d47365b1bbaf3f781902f7c193008fad672b834f3e6bdf5fe3afecd6ae22d423 WatchSource:0}: Error finding container d47365b1bbaf3f781902f7c193008fad672b834f3e6bdf5fe3afecd6ae22d423: Status 404 returned error can't find the container with id d47365b1bbaf3f781902f7c193008fad672b834f3e6bdf5fe3afecd6ae22d423 Sep 08 02:26:36.430865 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:36.430831 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" event={"ID":"2414c1ef-e298-49fd-a86a-557585b4b622","Type":"ContainerStarted","Data":"d47365b1bbaf3f781902f7c193008fad672b834f3e6bdf5fe3afecd6ae22d423"} Sep 08 02:26:38.206271 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:38.206229 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-d6lrq"] Sep 08 02:26:38.209368 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:38.209346 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d6lrq" Sep 08 02:26:38.212699 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:38.212677 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 08 02:26:38.212805 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:38.212730 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 08 02:26:38.214040 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:38.214017 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-wsclz\"" Sep 08 02:26:38.219279 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:38.219258 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-d6lrq"] Sep 08 02:26:38.338269 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:38.338234 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/28cbda96-49b9-4abe-9115-cbf33eac7123-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-d6lrq\" (UID: \"28cbda96-49b9-4abe-9115-cbf33eac7123\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d6lrq" Sep 08 02:26:38.338433 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:38.338292 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv7rk\" (UniqueName: \"kubernetes.io/projected/28cbda96-49b9-4abe-9115-cbf33eac7123-kube-api-access-rv7rk\") pod \"cert-manager-cainjector-559cdd8c87-d6lrq\" (UID: \"28cbda96-49b9-4abe-9115-cbf33eac7123\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d6lrq" Sep 08 02:26:38.439323 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:38.439287 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/28cbda96-49b9-4abe-9115-cbf33eac7123-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-d6lrq\" (UID: \"28cbda96-49b9-4abe-9115-cbf33eac7123\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d6lrq" Sep 08 02:26:38.439526 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:38.439342 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv7rk\" (UniqueName: \"kubernetes.io/projected/28cbda96-49b9-4abe-9115-cbf33eac7123-kube-api-access-rv7rk\") pod \"cert-manager-cainjector-559cdd8c87-d6lrq\" (UID: \"28cbda96-49b9-4abe-9115-cbf33eac7123\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d6lrq" Sep 08 02:26:38.450554 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:38.450519 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/28cbda96-49b9-4abe-9115-cbf33eac7123-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-d6lrq\" (UID: \"28cbda96-49b9-4abe-9115-cbf33eac7123\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d6lrq" Sep 08 02:26:38.450651 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:38.450562 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv7rk\" (UniqueName: \"kubernetes.io/projected/28cbda96-49b9-4abe-9115-cbf33eac7123-kube-api-access-rv7rk\") pod \"cert-manager-cainjector-559cdd8c87-d6lrq\" (UID: \"28cbda96-49b9-4abe-9115-cbf33eac7123\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d6lrq" Sep 08 02:26:38.532315 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:38.532236 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d6lrq" Sep 08 02:26:38.914215 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:38.914182 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-d6lrq"] Sep 08 02:26:38.916546 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:26:38.916518 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28cbda96_49b9_4abe_9115_cbf33eac7123.slice/crio-05682789ea4245659ebb24164fa15156c28a0fc9663ffda3c3594816f0e5644c WatchSource:0}: Error finding container 05682789ea4245659ebb24164fa15156c28a0fc9663ffda3c3594816f0e5644c: Status 404 returned error can't find the container with id 05682789ea4245659ebb24164fa15156c28a0fc9663ffda3c3594816f0e5644c Sep 08 02:26:39.450156 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:39.450119 2560 generic.go:356] "Generic (PLEG): container finished" podID="2414c1ef-e298-49fd-a86a-557585b4b622" containerID="570c2f4bbf788f51c6a2f022d913d9ae2e75d8cf00aaeb3890fe302e3e577d1b" exitCode=0 Sep 08 02:26:39.450625 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:39.450211 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" event={"ID":"2414c1ef-e298-49fd-a86a-557585b4b622","Type":"ContainerDied","Data":"570c2f4bbf788f51c6a2f022d913d9ae2e75d8cf00aaeb3890fe302e3e577d1b"} Sep 08 02:26:39.452166 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:39.452135 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d6lrq" event={"ID":"28cbda96-49b9-4abe-9115-cbf33eac7123","Type":"ContainerStarted","Data":"05682789ea4245659ebb24164fa15156c28a0fc9663ffda3c3594816f0e5644c"} Sep 08 02:26:40.461081 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:40.461042 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" event={"ID":"2414c1ef-e298-49fd-a86a-557585b4b622","Type":"ContainerStarted","Data":"f9dc4a82c4f9d139906f53aaa1c7c47d93160ad5bcd91fbadcc327e6ab17f947"} Sep 08 02:26:40.461545 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:40.461261 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" Sep 08 02:26:40.486093 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:40.486042 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" podStartSLOduration=1.951443538 podStartE2EDuration="5.486028183s" totalImagesPullingTime="3.104555831s" totalInitContainerRuntime="430.028814ms" isStatefulPod=false podCreationTimestamp="2026-09-08 02:26:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:26:40.48414188 +0000 UTC m=+296.015786736" watchObservedRunningTime="2026-09-08 02:26:40.486028183 +0000 UTC m=+296.017673048" Sep 08 02:26:40.977762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:40.977727 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-2h9pg"] Sep 08 02:26:40.980079 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:40.980051 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-2h9pg" Sep 08 02:26:40.983746 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:40.983718 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-s28qw\"" Sep 08 02:26:40.994935 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:40.994684 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-2h9pg"] Sep 08 02:26:41.165552 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:41.165516 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e88f05f6-61d7-404b-b5c6-efb35fc0288c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-2h9pg\" (UID: \"e88f05f6-61d7-404b-b5c6-efb35fc0288c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2h9pg" Sep 08 02:26:41.165552 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:41.165556 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbkjk\" (UniqueName: \"kubernetes.io/projected/e88f05f6-61d7-404b-b5c6-efb35fc0288c-kube-api-access-wbkjk\") pod \"cert-manager-webhook-6486ffd997-2h9pg\" (UID: \"e88f05f6-61d7-404b-b5c6-efb35fc0288c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2h9pg" Sep 08 02:26:41.266351 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:41.266255 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e88f05f6-61d7-404b-b5c6-efb35fc0288c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-2h9pg\" (UID: \"e88f05f6-61d7-404b-b5c6-efb35fc0288c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2h9pg" Sep 08 02:26:41.266351 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:41.266306 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbkjk\" (UniqueName: \"kubernetes.io/projected/e88f05f6-61d7-404b-b5c6-efb35fc0288c-kube-api-access-wbkjk\") pod \"cert-manager-webhook-6486ffd997-2h9pg\" (UID: \"e88f05f6-61d7-404b-b5c6-efb35fc0288c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2h9pg" Sep 08 02:26:41.281585 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:41.281553 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e88f05f6-61d7-404b-b5c6-efb35fc0288c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-2h9pg\" (UID: \"e88f05f6-61d7-404b-b5c6-efb35fc0288c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2h9pg" Sep 08 02:26:41.281725 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:41.281641 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbkjk\" (UniqueName: \"kubernetes.io/projected/e88f05f6-61d7-404b-b5c6-efb35fc0288c-kube-api-access-wbkjk\") pod \"cert-manager-webhook-6486ffd997-2h9pg\" (UID: \"e88f05f6-61d7-404b-b5c6-efb35fc0288c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-2h9pg" Sep 08 02:26:41.300382 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:41.300347 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-2h9pg" Sep 08 02:26:42.015978 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:42.015953 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-2h9pg"] Sep 08 02:26:42.018267 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:26:42.018241 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode88f05f6_61d7_404b_b5c6_efb35fc0288c.slice/crio-9d9557c0f34c2f3b9c13c0f9ad1ca183b8f8323a7ffce8d22d64c0183f2cf342 WatchSource:0}: Error finding container 9d9557c0f34c2f3b9c13c0f9ad1ca183b8f8323a7ffce8d22d64c0183f2cf342: Status 404 returned error can't find the container with id 9d9557c0f34c2f3b9c13c0f9ad1ca183b8f8323a7ffce8d22d64c0183f2cf342 Sep 08 02:26:42.473197 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:42.473145 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d6lrq" event={"ID":"28cbda96-49b9-4abe-9115-cbf33eac7123","Type":"ContainerStarted","Data":"f6143222a26daaa66edd6aab73fabaef4b99e934eafd6691e55b85255af97473"} Sep 08 02:26:42.474696 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:42.474673 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-2h9pg" event={"ID":"e88f05f6-61d7-404b-b5c6-efb35fc0288c","Type":"ContainerStarted","Data":"0e939f6c610135fdc205a5f51ab8733957f1a4ab0f089c6ee7f1f73c0fb5e2e4"} Sep 08 02:26:42.474696 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:42.474701 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-2h9pg" event={"ID":"e88f05f6-61d7-404b-b5c6-efb35fc0288c","Type":"ContainerStarted","Data":"9d9557c0f34c2f3b9c13c0f9ad1ca183b8f8323a7ffce8d22d64c0183f2cf342"} Sep 08 02:26:42.474854 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:42.474781 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-2h9pg" Sep 08 02:26:42.494358 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:42.494318 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d6lrq" podStartSLOduration=1.62835595 podStartE2EDuration="4.494305697s" totalImagesPullingTime="2.865949747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:26:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:26:42.491949605 +0000 UTC m=+298.023594469" watchObservedRunningTime="2026-09-08 02:26:42.494305697 +0000 UTC m=+298.025950561" Sep 08 02:26:42.519181 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:42.519130 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-2h9pg" podStartSLOduration=2.519115712 podStartE2EDuration="2.519115712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:26:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:26:42.517829945 +0000 UTC m=+298.049474811" watchObservedRunningTime="2026-09-08 02:26:42.519115712 +0000 UTC m=+298.050760577" Sep 08 02:26:44.919034 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:44.919003 2560 kubelet.go:1648] "Image garbage collection succeeded" Sep 08 02:26:48.483742 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:48.483712 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-2h9pg" Sep 08 02:26:51.468319 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:26:51.468277 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-5c89fb8b78-qjpw2" Sep 08 02:27:00.210352 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:00.210312 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-65f559df56-cqzn5"] Sep 08 02:27:00.217557 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:00.217532 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65f559df56-cqzn5" Sep 08 02:27:00.220902 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:00.220871 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 08 02:27:00.221184 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:00.221166 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 08 02:27:00.221489 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:00.221449 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-bndfb\"" Sep 08 02:27:00.227204 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:00.227182 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65f559df56-cqzn5"] Sep 08 02:27:00.314027 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:00.313988 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfqxw\" (UniqueName: \"kubernetes.io/projected/7b7b6a75-411c-41dd-b5fe-e61ff6172318-kube-api-access-pfqxw\") pod \"kube-auth-proxy-65f559df56-cqzn5\" (UID: \"7b7b6a75-411c-41dd-b5fe-e61ff6172318\") " pod="openshift-ingress/kube-auth-proxy-65f559df56-cqzn5" Sep 08 02:27:00.314027 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:00.314029 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b7b6a75-411c-41dd-b5fe-e61ff6172318-tls-certs\") pod \"kube-auth-proxy-65f559df56-cqzn5\" (UID: \"7b7b6a75-411c-41dd-b5fe-e61ff6172318\") " pod="openshift-ingress/kube-auth-proxy-65f559df56-cqzn5" Sep 08 02:27:00.314238 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:00.314047 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b7b6a75-411c-41dd-b5fe-e61ff6172318-tmp\") pod \"kube-auth-proxy-65f559df56-cqzn5\" (UID: \"7b7b6a75-411c-41dd-b5fe-e61ff6172318\") " pod="openshift-ingress/kube-auth-proxy-65f559df56-cqzn5" Sep 08 02:27:00.414907 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:00.414869 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfqxw\" (UniqueName: \"kubernetes.io/projected/7b7b6a75-411c-41dd-b5fe-e61ff6172318-kube-api-access-pfqxw\") pod \"kube-auth-proxy-65f559df56-cqzn5\" (UID: \"7b7b6a75-411c-41dd-b5fe-e61ff6172318\") " pod="openshift-ingress/kube-auth-proxy-65f559df56-cqzn5" Sep 08 02:27:00.415093 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:00.414916 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b7b6a75-411c-41dd-b5fe-e61ff6172318-tls-certs\") pod \"kube-auth-proxy-65f559df56-cqzn5\" (UID: \"7b7b6a75-411c-41dd-b5fe-e61ff6172318\") " pod="openshift-ingress/kube-auth-proxy-65f559df56-cqzn5" Sep 08 02:27:00.415093 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:00.414934 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b7b6a75-411c-41dd-b5fe-e61ff6172318-tmp\") pod \"kube-auth-proxy-65f559df56-cqzn5\" (UID: \"7b7b6a75-411c-41dd-b5fe-e61ff6172318\") " pod="openshift-ingress/kube-auth-proxy-65f559df56-cqzn5" Sep 08 02:27:00.417438 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:00.417414 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b7b6a75-411c-41dd-b5fe-e61ff6172318-tmp\") pod \"kube-auth-proxy-65f559df56-cqzn5\" (UID: \"7b7b6a75-411c-41dd-b5fe-e61ff6172318\") " pod="openshift-ingress/kube-auth-proxy-65f559df56-cqzn5" Sep 08 02:27:00.417571 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:00.417554 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b7b6a75-411c-41dd-b5fe-e61ff6172318-tls-certs\") pod \"kube-auth-proxy-65f559df56-cqzn5\" (UID: \"7b7b6a75-411c-41dd-b5fe-e61ff6172318\") " pod="openshift-ingress/kube-auth-proxy-65f559df56-cqzn5" Sep 08 02:27:00.442347 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:00.442326 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfqxw\" (UniqueName: \"kubernetes.io/projected/7b7b6a75-411c-41dd-b5fe-e61ff6172318-kube-api-access-pfqxw\") pod \"kube-auth-proxy-65f559df56-cqzn5\" (UID: \"7b7b6a75-411c-41dd-b5fe-e61ff6172318\") " pod="openshift-ingress/kube-auth-proxy-65f559df56-cqzn5" Sep 08 02:27:00.529991 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:00.529919 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65f559df56-cqzn5" Sep 08 02:27:00.804495 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:00.804455 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65f559df56-cqzn5"] Sep 08 02:27:00.806438 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:27:00.806413 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b7b6a75_411c_41dd_b5fe_e61ff6172318.slice/crio-8826c98f9f4a23a32c271cd893d30834c849a8d734c3135eac433a409d5b707e WatchSource:0}: Error finding container 8826c98f9f4a23a32c271cd893d30834c849a8d734c3135eac433a409d5b707e: Status 404 returned error can't find the container with id 8826c98f9f4a23a32c271cd893d30834c849a8d734c3135eac433a409d5b707e Sep 08 02:27:00.808364 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:00.808348 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 02:27:01.561624 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:01.561583 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65f559df56-cqzn5" event={"ID":"7b7b6a75-411c-41dd-b5fe-e61ff6172318","Type":"ContainerStarted","Data":"8826c98f9f4a23a32c271cd893d30834c849a8d734c3135eac433a409d5b707e"} Sep 08 02:27:01.584823 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:01.584722 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-54765d59fd-rvkzh"] Sep 08 02:27:01.587504 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:01.587453 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-54765d59fd-rvkzh" Sep 08 02:27:01.591005 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:01.590979 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 08 02:27:01.592026 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:01.591857 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 08 02:27:01.592026 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:01.591891 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-8nkr6\"" Sep 08 02:27:01.598182 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:01.598008 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\"" Sep 08 02:27:01.601760 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:01.601736 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-54765d59fd-rvkzh"] Sep 08 02:27:01.624531 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:01.624489 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt24b\" (UniqueName: \"kubernetes.io/projected/dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435-kube-api-access-wt24b\") pod \"kserve-module-controller-manager-54765d59fd-rvkzh\" (UID: \"dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-rvkzh" Sep 08 02:27:01.624704 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:01.624573 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435-manifests\") pod \"kserve-module-controller-manager-54765d59fd-rvkzh\" (UID: \"dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-rvkzh" Sep 08 02:27:01.624704 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:01.624604 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435-tmp\") pod \"kserve-module-controller-manager-54765d59fd-rvkzh\" (UID: \"dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-rvkzh" Sep 08 02:27:01.624704 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:01.624697 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435-trusted-ca\") pod \"kserve-module-controller-manager-54765d59fd-rvkzh\" (UID: \"dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-rvkzh" Sep 08 02:27:01.726828 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:01.726188 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435-trusted-ca\") pod \"kserve-module-controller-manager-54765d59fd-rvkzh\" (UID: \"dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-rvkzh" Sep 08 02:27:01.726828 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:01.726270 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt24b\" (UniqueName: \"kubernetes.io/projected/dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435-kube-api-access-wt24b\") pod \"kserve-module-controller-manager-54765d59fd-rvkzh\" (UID: \"dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-rvkzh" Sep 08 02:27:01.726828 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:01.726311 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435-manifests\") pod \"kserve-module-controller-manager-54765d59fd-rvkzh\" (UID: \"dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-rvkzh" Sep 08 02:27:01.726828 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:01.726336 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435-tmp\") pod \"kserve-module-controller-manager-54765d59fd-rvkzh\" (UID: \"dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-rvkzh" Sep 08 02:27:01.726828 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:01.726781 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435-tmp\") pod \"kserve-module-controller-manager-54765d59fd-rvkzh\" (UID: \"dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-rvkzh" Sep 08 02:27:01.727207 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:01.727033 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435-manifests\") pod \"kserve-module-controller-manager-54765d59fd-rvkzh\" (UID: \"dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-rvkzh" Sep 08 02:27:01.727696 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:01.727650 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435-trusted-ca\") pod \"kserve-module-controller-manager-54765d59fd-rvkzh\" (UID: \"dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-rvkzh" Sep 08 02:27:01.745150 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:01.745097 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt24b\" (UniqueName: \"kubernetes.io/projected/dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435-kube-api-access-wt24b\") pod \"kserve-module-controller-manager-54765d59fd-rvkzh\" (UID: \"dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435\") " pod="opendatahub/kserve-module-controller-manager-54765d59fd-rvkzh" Sep 08 02:27:01.903509 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:01.903346 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-54765d59fd-rvkzh" Sep 08 02:27:02.210846 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:02.210758 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-54765d59fd-rvkzh"] Sep 08 02:27:02.216034 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:27:02.216004 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc55461f_bb4a_4cf8_aa2a_0f7ebd6a5435.slice/crio-615fd288ac5f2c3d5c9434a72444e7b30711da28b56b68d0b097d897b4d73bc1 WatchSource:0}: Error finding container 615fd288ac5f2c3d5c9434a72444e7b30711da28b56b68d0b097d897b4d73bc1: Status 404 returned error can't find the container with id 615fd288ac5f2c3d5c9434a72444e7b30711da28b56b68d0b097d897b4d73bc1 Sep 08 02:27:02.439397 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:02.439364 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-ffcddf5cd-5zb86"] Sep 08 02:27:02.445670 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:02.445642 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-ffcddf5cd-5zb86" Sep 08 02:27:02.448848 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:02.448821 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Sep 08 02:27:02.448986 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:02.448848 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 08 02:27:02.448986 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:02.448819 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 08 02:27:02.449259 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:02.449241 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-jw4d5\"" Sep 08 02:27:02.458117 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:02.458090 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-ffcddf5cd-5zb86"] Sep 08 02:27:02.533206 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:02.533110 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk8jt\" (UniqueName: \"kubernetes.io/projected/30ab6d94-b77f-4c9c-9d7d-a29fc93bc6b3-kube-api-access-dk8jt\") pod \"dns-operator-controller-manager-ffcddf5cd-5zb86\" (UID: \"30ab6d94-b77f-4c9c-9d7d-a29fc93bc6b3\") " pod="kuadrant-system/dns-operator-controller-manager-ffcddf5cd-5zb86" Sep 08 02:27:02.568315 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:02.568272 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-54765d59fd-rvkzh" event={"ID":"dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435","Type":"ContainerStarted","Data":"615fd288ac5f2c3d5c9434a72444e7b30711da28b56b68d0b097d897b4d73bc1"} Sep 08 02:27:02.633909 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:02.633870 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk8jt\" (UniqueName: \"kubernetes.io/projected/30ab6d94-b77f-4c9c-9d7d-a29fc93bc6b3-kube-api-access-dk8jt\") pod \"dns-operator-controller-manager-ffcddf5cd-5zb86\" (UID: \"30ab6d94-b77f-4c9c-9d7d-a29fc93bc6b3\") " pod="kuadrant-system/dns-operator-controller-manager-ffcddf5cd-5zb86" Sep 08 02:27:02.651766 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:02.651735 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk8jt\" (UniqueName: \"kubernetes.io/projected/30ab6d94-b77f-4c9c-9d7d-a29fc93bc6b3-kube-api-access-dk8jt\") pod \"dns-operator-controller-manager-ffcddf5cd-5zb86\" (UID: \"30ab6d94-b77f-4c9c-9d7d-a29fc93bc6b3\") " pod="kuadrant-system/dns-operator-controller-manager-ffcddf5cd-5zb86" Sep 08 02:27:02.760258 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:02.760220 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-ffcddf5cd-5zb86" Sep 08 02:27:03.520627 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:03.520597 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-ffcddf5cd-5zb86"] Sep 08 02:27:03.522980 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:27:03.522938 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30ab6d94_b77f_4c9c_9d7d_a29fc93bc6b3.slice/crio-ae9dfa0a5e7218f4a0a3582cf78a0739af8e9c797d799209846085e28aca5cf4 WatchSource:0}: Error finding container ae9dfa0a5e7218f4a0a3582cf78a0739af8e9c797d799209846085e28aca5cf4: Status 404 returned error can't find the container with id ae9dfa0a5e7218f4a0a3582cf78a0739af8e9c797d799209846085e28aca5cf4 Sep 08 02:27:03.575989 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:03.575942 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-ffcddf5cd-5zb86" event={"ID":"30ab6d94-b77f-4c9c-9d7d-a29fc93bc6b3","Type":"ContainerStarted","Data":"ae9dfa0a5e7218f4a0a3582cf78a0739af8e9c797d799209846085e28aca5cf4"} Sep 08 02:27:03.577752 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:03.577718 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65f559df56-cqzn5" event={"ID":"7b7b6a75-411c-41dd-b5fe-e61ff6172318","Type":"ContainerStarted","Data":"efeef3645c4b695483cc1e33b32bc2425881ada888dc6ff56e0eb6bdbb127960"} Sep 08 02:27:03.617123 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:03.617077 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-65f559df56-cqzn5" podStartSLOduration=1.152588406 podStartE2EDuration="3.617062674s" totalImagesPullingTime="2.464474268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:27:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:27:03.615297614 +0000 UTC m=+319.146942481" watchObservedRunningTime="2026-09-08 02:27:03.617062674 +0000 UTC m=+319.148707539" Sep 08 02:27:04.356844 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.356811 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-tx7kp"] Sep 08 02:27:04.360102 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.360083 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-tx7kp" Sep 08 02:27:04.363547 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.363528 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 08 02:27:04.363728 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.363708 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jrgr6\"" Sep 08 02:27:04.370760 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.370737 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-tx7kp"] Sep 08 02:27:04.443609 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.443587 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-tx7kp"] Sep 08 02:27:04.450368 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.450346 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqpwc\" (UniqueName: \"kubernetes.io/projected/f35c8ea3-168f-40fe-94e1-3b2cd22156fd-kube-api-access-kqpwc\") pod \"limitador-limitador-859c4d96d6-tx7kp\" (UID: \"f35c8ea3-168f-40fe-94e1-3b2cd22156fd\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-tx7kp" Sep 08 02:27:04.450485 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.450401 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f35c8ea3-168f-40fe-94e1-3b2cd22156fd-config-file\") pod \"limitador-limitador-859c4d96d6-tx7kp\" (UID: \"f35c8ea3-168f-40fe-94e1-3b2cd22156fd\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-tx7kp" Sep 08 02:27:04.551231 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.551196 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f35c8ea3-168f-40fe-94e1-3b2cd22156fd-config-file\") pod \"limitador-limitador-859c4d96d6-tx7kp\" (UID: \"f35c8ea3-168f-40fe-94e1-3b2cd22156fd\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-tx7kp" Sep 08 02:27:04.551408 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.551271 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqpwc\" (UniqueName: \"kubernetes.io/projected/f35c8ea3-168f-40fe-94e1-3b2cd22156fd-kube-api-access-kqpwc\") pod \"limitador-limitador-859c4d96d6-tx7kp\" (UID: \"f35c8ea3-168f-40fe-94e1-3b2cd22156fd\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-tx7kp" Sep 08 02:27:04.551833 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.551812 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f35c8ea3-168f-40fe-94e1-3b2cd22156fd-config-file\") pod \"limitador-limitador-859c4d96d6-tx7kp\" (UID: \"f35c8ea3-168f-40fe-94e1-3b2cd22156fd\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-tx7kp" Sep 08 02:27:04.563508 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.563482 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqpwc\" (UniqueName: \"kubernetes.io/projected/f35c8ea3-168f-40fe-94e1-3b2cd22156fd-kube-api-access-kqpwc\") pod \"limitador-limitador-859c4d96d6-tx7kp\" (UID: \"f35c8ea3-168f-40fe-94e1-3b2cd22156fd\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-tx7kp" Sep 08 02:27:04.584991 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.584952 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-54765d59fd-rvkzh" event={"ID":"dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435","Type":"ContainerStarted","Data":"047d631450577a68bef28f042c74d0ff579a8abd7746d50568904f72c4ce40b3"} Sep 08 02:27:04.585335 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.585014 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-54765d59fd-rvkzh" Sep 08 02:27:04.612513 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.612427 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-54765d59fd-rvkzh" podStartSLOduration=1.439890149 podStartE2EDuration="3.612410352s" totalImagesPullingTime="2.172520203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:27:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:27:04.612046598 +0000 UTC m=+320.143691464" watchObservedRunningTime="2026-09-08 02:27:04.612410352 +0000 UTC m=+320.144055217" Sep 08 02:27:04.671557 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.671524 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-tx7kp" Sep 08 02:27:04.782532 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.782492 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv"] Sep 08 02:27:04.785236 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.785208 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv" Sep 08 02:27:04.795606 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.795575 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv"] Sep 08 02:27:04.818186 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.818112 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv"] Sep 08 02:27:04.856982 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.854061 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txfml\" (UniqueName: \"kubernetes.io/projected/f6c0f248-c2cd-4596-961c-606065a8deef-kube-api-access-txfml\") pod \"limitador-limitador-85d9db4d5d-7zwzv\" (UID: \"f6c0f248-c2cd-4596-961c-606065a8deef\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv" Sep 08 02:27:04.856982 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.854115 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f6c0f248-c2cd-4596-961c-606065a8deef-config-file\") pod \"limitador-limitador-85d9db4d5d-7zwzv\" (UID: \"f6c0f248-c2cd-4596-961c-606065a8deef\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv" Sep 08 02:27:04.945659 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.945346 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-tx7kp"] Sep 08 02:27:04.949823 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:27:04.949792 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf35c8ea3_168f_40fe_94e1_3b2cd22156fd.slice/crio-de8e3b785e82370ce896f809311a22760b2ebb644037294023156814a82a67a4 WatchSource:0}: Error finding container de8e3b785e82370ce896f809311a22760b2ebb644037294023156814a82a67a4: Status 404 returned error can't find the container with id de8e3b785e82370ce896f809311a22760b2ebb644037294023156814a82a67a4 Sep 08 02:27:04.956534 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.955425 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txfml\" (UniqueName: \"kubernetes.io/projected/f6c0f248-c2cd-4596-961c-606065a8deef-kube-api-access-txfml\") pod \"limitador-limitador-85d9db4d5d-7zwzv\" (UID: \"f6c0f248-c2cd-4596-961c-606065a8deef\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv" Sep 08 02:27:04.956534 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.955563 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f6c0f248-c2cd-4596-961c-606065a8deef-config-file\") pod \"limitador-limitador-85d9db4d5d-7zwzv\" (UID: \"f6c0f248-c2cd-4596-961c-606065a8deef\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv" Sep 08 02:27:04.956534 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.956450 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f6c0f248-c2cd-4596-961c-606065a8deef-config-file\") pod \"limitador-limitador-85d9db4d5d-7zwzv\" (UID: \"f6c0f248-c2cd-4596-961c-606065a8deef\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv" Sep 08 02:27:04.964655 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:04.964626 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txfml\" (UniqueName: \"kubernetes.io/projected/f6c0f248-c2cd-4596-961c-606065a8deef-kube-api-access-txfml\") pod \"limitador-limitador-85d9db4d5d-7zwzv\" (UID: \"f6c0f248-c2cd-4596-961c-606065a8deef\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv" Sep 08 02:27:05.099507 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:05.099449 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv" Sep 08 02:27:05.394949 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:05.394817 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv"] Sep 08 02:27:05.397256 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:27:05.397223 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6c0f248_c2cd_4596_961c_606065a8deef.slice/crio-82089a50f579970b873a80714ad84f7298be88d11bbe87e318e73a5f5517a2f4 WatchSource:0}: Error finding container 82089a50f579970b873a80714ad84f7298be88d11bbe87e318e73a5f5517a2f4: Status 404 returned error can't find the container with id 82089a50f579970b873a80714ad84f7298be88d11bbe87e318e73a5f5517a2f4 Sep 08 02:27:05.592198 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:05.592162 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-tx7kp" event={"ID":"f35c8ea3-168f-40fe-94e1-3b2cd22156fd","Type":"ContainerStarted","Data":"de8e3b785e82370ce896f809311a22760b2ebb644037294023156814a82a67a4"} Sep 08 02:27:05.594160 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:05.594131 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv" event={"ID":"f6c0f248-c2cd-4596-961c-606065a8deef","Type":"ContainerStarted","Data":"82089a50f579970b873a80714ad84f7298be88d11bbe87e318e73a5f5517a2f4"} Sep 08 02:27:07.609023 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:07.608987 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv" event={"ID":"f6c0f248-c2cd-4596-961c-606065a8deef","Type":"ContainerStarted","Data":"7206e19ce9998b1012685a0e09c153422c48ff331620a8d1586d5741b8b7b1ee"} Sep 08 02:27:07.609613 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:07.609087 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv" Sep 08 02:27:07.610771 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:07.610746 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-ffcddf5cd-5zb86" event={"ID":"30ab6d94-b77f-4c9c-9d7d-a29fc93bc6b3","Type":"ContainerStarted","Data":"78386d19ede94af220b7618a48c3932733e1e8f066cd6b1d5a899ef44b97cad5"} Sep 08 02:27:07.610878 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:07.610858 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-ffcddf5cd-5zb86" Sep 08 02:27:07.612304 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:07.612282 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-tx7kp" event={"ID":"f35c8ea3-168f-40fe-94e1-3b2cd22156fd","Type":"ContainerStarted","Data":"004ef9ba9e697d6005a382ef778f71b9d58ea4c909068c8422b7bc80959fefad"} Sep 08 02:27:07.612394 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:07.612336 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-tx7kp" Sep 08 02:27:07.629539 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:07.629493 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv" podStartSLOduration=1.946861279 podStartE2EDuration="3.629479927s" totalImagesPullingTime="1.682618648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:27:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:27:07.627727943 +0000 UTC m=+323.159372811" watchObservedRunningTime="2026-09-08 02:27:07.629479927 +0000 UTC m=+323.161124789" Sep 08 02:27:07.647233 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:07.647178 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-ffcddf5cd-5zb86" podStartSLOduration=2.631369576 podStartE2EDuration="5.647165271s" totalImagesPullingTime="3.015795695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:27:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:27:07.646179634 +0000 UTC m=+323.177824510" watchObservedRunningTime="2026-09-08 02:27:07.647165271 +0000 UTC m=+323.178810136" Sep 08 02:27:07.666435 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:07.666389 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-tx7kp" podStartSLOduration=1.5343306220000001 podStartE2EDuration="3.666374284s" totalImagesPullingTime="2.132043662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:27:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:27:07.665013259 +0000 UTC m=+323.196658124" watchObservedRunningTime="2026-09-08 02:27:07.666374284 +0000 UTC m=+323.198019148" Sep 08 02:27:18.617833 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:18.617798 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-tx7kp" Sep 08 02:27:18.618277 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:18.617866 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv" Sep 08 02:27:18.619158 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:18.619135 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-ffcddf5cd-5zb86" Sep 08 02:27:18.692568 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:18.692534 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-tx7kp"] Sep 08 02:27:18.700454 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:18.700402 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-tx7kp" podUID="f35c8ea3-168f-40fe-94e1-3b2cd22156fd" containerName="limitador" containerID="cri-o://004ef9ba9e697d6005a382ef778f71b9d58ea4c909068c8422b7bc80959fefad" gracePeriod=30 Sep 08 02:27:19.656334 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.656308 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-tx7kp" Sep 08 02:27:19.672208 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.672174 2560 generic.go:356] "Generic (PLEG): container finished" podID="f35c8ea3-168f-40fe-94e1-3b2cd22156fd" containerID="004ef9ba9e697d6005a382ef778f71b9d58ea4c909068c8422b7bc80959fefad" exitCode=0 Sep 08 02:27:19.672342 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.672218 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-tx7kp" Sep 08 02:27:19.672342 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.672252 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-tx7kp" event={"ID":"f35c8ea3-168f-40fe-94e1-3b2cd22156fd","Type":"ContainerDied","Data":"004ef9ba9e697d6005a382ef778f71b9d58ea4c909068c8422b7bc80959fefad"} Sep 08 02:27:19.672342 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.672309 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-tx7kp" event={"ID":"f35c8ea3-168f-40fe-94e1-3b2cd22156fd","Type":"ContainerDied","Data":"de8e3b785e82370ce896f809311a22760b2ebb644037294023156814a82a67a4"} Sep 08 02:27:19.672456 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.672352 2560 scope.go:117] "RemoveContainer" containerID="004ef9ba9e697d6005a382ef778f71b9d58ea4c909068c8422b7bc80959fefad" Sep 08 02:27:19.682991 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.682972 2560 scope.go:117] "RemoveContainer" containerID="004ef9ba9e697d6005a382ef778f71b9d58ea4c909068c8422b7bc80959fefad" Sep 08 02:27:19.683353 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:27:19.683329 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"004ef9ba9e697d6005a382ef778f71b9d58ea4c909068c8422b7bc80959fefad\": container with ID starting with 004ef9ba9e697d6005a382ef778f71b9d58ea4c909068c8422b7bc80959fefad not found: ID does not exist" containerID="004ef9ba9e697d6005a382ef778f71b9d58ea4c909068c8422b7bc80959fefad" Sep 08 02:27:19.683444 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.683362 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"004ef9ba9e697d6005a382ef778f71b9d58ea4c909068c8422b7bc80959fefad"} err="failed to get container status \"004ef9ba9e697d6005a382ef778f71b9d58ea4c909068c8422b7bc80959fefad\": rpc error: code = NotFound desc = could not find container \"004ef9ba9e697d6005a382ef778f71b9d58ea4c909068c8422b7bc80959fefad\": container with ID starting with 004ef9ba9e697d6005a382ef778f71b9d58ea4c909068c8422b7bc80959fefad not found: ID does not exist" Sep 08 02:27:19.788116 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.788015 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqpwc\" (UniqueName: \"kubernetes.io/projected/f35c8ea3-168f-40fe-94e1-3b2cd22156fd-kube-api-access-kqpwc\") pod \"f35c8ea3-168f-40fe-94e1-3b2cd22156fd\" (UID: \"f35c8ea3-168f-40fe-94e1-3b2cd22156fd\") " Sep 08 02:27:19.788116 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.788085 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f35c8ea3-168f-40fe-94e1-3b2cd22156fd-config-file\") pod \"f35c8ea3-168f-40fe-94e1-3b2cd22156fd\" (UID: \"f35c8ea3-168f-40fe-94e1-3b2cd22156fd\") " Sep 08 02:27:19.788528 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.788502 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f35c8ea3-168f-40fe-94e1-3b2cd22156fd-config-file" (OuterVolumeSpecName: "config-file") pod "f35c8ea3-168f-40fe-94e1-3b2cd22156fd" (UID: "f35c8ea3-168f-40fe-94e1-3b2cd22156fd"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 02:27:19.790355 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.790325 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f35c8ea3-168f-40fe-94e1-3b2cd22156fd-kube-api-access-kqpwc" (OuterVolumeSpecName: "kube-api-access-kqpwc") pod "f35c8ea3-168f-40fe-94e1-3b2cd22156fd" (UID: "f35c8ea3-168f-40fe-94e1-3b2cd22156fd"). InnerVolumeSpecName "kube-api-access-kqpwc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 02:27:19.863046 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.863014 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-657f59599b-dpqc5"] Sep 08 02:27:19.863369 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.863356 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f35c8ea3-168f-40fe-94e1-3b2cd22156fd" containerName="limitador" Sep 08 02:27:19.863369 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.863369 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f35c8ea3-168f-40fe-94e1-3b2cd22156fd" containerName="limitador" Sep 08 02:27:19.863477 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.863437 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f35c8ea3-168f-40fe-94e1-3b2cd22156fd" containerName="limitador" Sep 08 02:27:19.867567 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.867551 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-657f59599b-dpqc5" Sep 08 02:27:19.870761 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.870736 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 08 02:27:19.870883 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.870824 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 08 02:27:19.870983 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.870966 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6fs2r\"" Sep 08 02:27:19.871351 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.871329 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 08 02:27:19.878914 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.878890 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-657f59599b-dpqc5"] Sep 08 02:27:19.889370 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.889346 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqpwc\" (UniqueName: \"kubernetes.io/projected/f35c8ea3-168f-40fe-94e1-3b2cd22156fd-kube-api-access-kqpwc\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Sep 08 02:27:19.889370 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.889370 2560 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f35c8ea3-168f-40fe-94e1-3b2cd22156fd-config-file\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Sep 08 02:27:19.990135 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.990102 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pwnf\" (UniqueName: \"kubernetes.io/projected/18bd4cc6-3225-4611-971f-611be27802ee-kube-api-access-7pwnf\") pod \"maas-controller-657f59599b-dpqc5\" (UID: \"18bd4cc6-3225-4611-971f-611be27802ee\") " pod="opendatahub/maas-controller-657f59599b-dpqc5" Sep 08 02:27:19.990311 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.990154 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18bd4cc6-3225-4611-971f-611be27802ee-webhook-cert\") pod \"maas-controller-657f59599b-dpqc5\" (UID: \"18bd4cc6-3225-4611-971f-611be27802ee\") " pod="opendatahub/maas-controller-657f59599b-dpqc5" Sep 08 02:27:19.990369 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.990323 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18bd4cc6-3225-4611-971f-611be27802ee-metrics-certs\") pod \"maas-controller-657f59599b-dpqc5\" (UID: \"18bd4cc6-3225-4611-971f-611be27802ee\") " pod="opendatahub/maas-controller-657f59599b-dpqc5" Sep 08 02:27:19.998631 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:19.998606 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-tx7kp"] Sep 08 02:27:20.005289 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:20.005261 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-tx7kp"] Sep 08 02:27:20.091228 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:20.091138 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18bd4cc6-3225-4611-971f-611be27802ee-metrics-certs\") pod \"maas-controller-657f59599b-dpqc5\" (UID: \"18bd4cc6-3225-4611-971f-611be27802ee\") " pod="opendatahub/maas-controller-657f59599b-dpqc5" Sep 08 02:27:20.091228 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:20.091196 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pwnf\" (UniqueName: \"kubernetes.io/projected/18bd4cc6-3225-4611-971f-611be27802ee-kube-api-access-7pwnf\") pod \"maas-controller-657f59599b-dpqc5\" (UID: \"18bd4cc6-3225-4611-971f-611be27802ee\") " pod="opendatahub/maas-controller-657f59599b-dpqc5" Sep 08 02:27:20.091457 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:20.091233 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18bd4cc6-3225-4611-971f-611be27802ee-webhook-cert\") pod \"maas-controller-657f59599b-dpqc5\" (UID: \"18bd4cc6-3225-4611-971f-611be27802ee\") " pod="opendatahub/maas-controller-657f59599b-dpqc5" Sep 08 02:27:20.093767 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:20.093743 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18bd4cc6-3225-4611-971f-611be27802ee-webhook-cert\") pod \"maas-controller-657f59599b-dpqc5\" (UID: \"18bd4cc6-3225-4611-971f-611be27802ee\") " pod="opendatahub/maas-controller-657f59599b-dpqc5" Sep 08 02:27:20.093874 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:20.093775 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18bd4cc6-3225-4611-971f-611be27802ee-metrics-certs\") pod \"maas-controller-657f59599b-dpqc5\" (UID: \"18bd4cc6-3225-4611-971f-611be27802ee\") " pod="opendatahub/maas-controller-657f59599b-dpqc5" Sep 08 02:27:20.100729 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:20.100707 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pwnf\" (UniqueName: \"kubernetes.io/projected/18bd4cc6-3225-4611-971f-611be27802ee-kube-api-access-7pwnf\") pod \"maas-controller-657f59599b-dpqc5\" (UID: \"18bd4cc6-3225-4611-971f-611be27802ee\") " pod="opendatahub/maas-controller-657f59599b-dpqc5" Sep 08 02:27:20.179851 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:20.179812 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-657f59599b-dpqc5" Sep 08 02:27:20.453061 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:20.452921 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-657f59599b-dpqc5"] Sep 08 02:27:20.455823 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:27:20.455793 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18bd4cc6_3225_4611_971f_611be27802ee.slice/crio-ac448a3bd9785478c5c7fddadd266c613bdae3c52847562f3c6fabe079505375 WatchSource:0}: Error finding container ac448a3bd9785478c5c7fddadd266c613bdae3c52847562f3c6fabe079505375: Status 404 returned error can't find the container with id ac448a3bd9785478c5c7fddadd266c613bdae3c52847562f3c6fabe079505375 Sep 08 02:27:20.679011 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:20.678965 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-657f59599b-dpqc5" event={"ID":"18bd4cc6-3225-4611-971f-611be27802ee","Type":"ContainerStarted","Data":"ac448a3bd9785478c5c7fddadd266c613bdae3c52847562f3c6fabe079505375"} Sep 08 02:27:21.016781 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:21.016748 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f35c8ea3-168f-40fe-94e1-3b2cd22156fd" path="/var/lib/kubelet/pods/f35c8ea3-168f-40fe-94e1-3b2cd22156fd/volumes" Sep 08 02:27:22.691900 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:22.691867 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-657f59599b-dpqc5" event={"ID":"18bd4cc6-3225-4611-971f-611be27802ee","Type":"ContainerStarted","Data":"24e32b20386b287d3c2c9f08dbfe0a9bb9f5264be8fab334ba2e66d69434c086"} Sep 08 02:27:22.692275 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:22.691926 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-657f59599b-dpqc5" Sep 08 02:27:22.743749 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:22.743695 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-657f59599b-dpqc5" podStartSLOduration=2.101259983 podStartE2EDuration="3.743680138s" totalImagesPullingTime="1.642420155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:27:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:27:22.742556516 +0000 UTC m=+338.274201380" watchObservedRunningTime="2026-09-08 02:27:22.743680138 +0000 UTC m=+338.275325002" Sep 08 02:27:25.595117 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:25.595082 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-54765d59fd-rvkzh" Sep 08 02:27:25.827917 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:25.827884 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-5grt4"] Sep 08 02:27:25.831401 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:25.831385 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-5grt4" Sep 08 02:27:25.834576 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:25.834554 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6krsj\"" Sep 08 02:27:25.841106 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:25.841081 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-5grt4"] Sep 08 02:27:25.948745 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:25.948713 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5jrt\" (UniqueName: \"kubernetes.io/projected/e58e0d3e-3dca-4cb1-9020-fcfa14abd346-kube-api-access-m5jrt\") pod \"authorino-b54df94b4-5grt4\" (UID: \"e58e0d3e-3dca-4cb1-9020-fcfa14abd346\") " pod="kuadrant-system/authorino-b54df94b4-5grt4" Sep 08 02:27:26.050134 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:26.050102 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5jrt\" (UniqueName: \"kubernetes.io/projected/e58e0d3e-3dca-4cb1-9020-fcfa14abd346-kube-api-access-m5jrt\") pod \"authorino-b54df94b4-5grt4\" (UID: \"e58e0d3e-3dca-4cb1-9020-fcfa14abd346\") " pod="kuadrant-system/authorino-b54df94b4-5grt4" Sep 08 02:27:26.057191 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:26.057163 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-q4nhg"] Sep 08 02:27:26.060360 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:26.060337 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5jrt\" (UniqueName: \"kubernetes.io/projected/e58e0d3e-3dca-4cb1-9020-fcfa14abd346-kube-api-access-m5jrt\") pod \"authorino-b54df94b4-5grt4\" (UID: \"e58e0d3e-3dca-4cb1-9020-fcfa14abd346\") " pod="kuadrant-system/authorino-b54df94b4-5grt4" Sep 08 02:27:26.060501 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:26.060486 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-q4nhg" Sep 08 02:27:26.072011 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:26.071988 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-q4nhg"] Sep 08 02:27:26.140651 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:26.140617 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-5grt4" Sep 08 02:27:26.150871 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:26.150835 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vjqd\" (UniqueName: \"kubernetes.io/projected/3c00376e-381e-43cb-8b44-c6c64323937b-kube-api-access-2vjqd\") pod \"authorino-c548b5d7d-q4nhg\" (UID: \"3c00376e-381e-43cb-8b44-c6c64323937b\") " pod="kuadrant-system/authorino-c548b5d7d-q4nhg" Sep 08 02:27:26.252111 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:26.252020 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vjqd\" (UniqueName: \"kubernetes.io/projected/3c00376e-381e-43cb-8b44-c6c64323937b-kube-api-access-2vjqd\") pod \"authorino-c548b5d7d-q4nhg\" (UID: \"3c00376e-381e-43cb-8b44-c6c64323937b\") " pod="kuadrant-system/authorino-c548b5d7d-q4nhg" Sep 08 02:27:26.261116 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:26.261090 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vjqd\" (UniqueName: \"kubernetes.io/projected/3c00376e-381e-43cb-8b44-c6c64323937b-kube-api-access-2vjqd\") pod \"authorino-c548b5d7d-q4nhg\" (UID: \"3c00376e-381e-43cb-8b44-c6c64323937b\") " pod="kuadrant-system/authorino-c548b5d7d-q4nhg" Sep 08 02:27:26.376167 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:26.376132 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-q4nhg" Sep 08 02:27:26.419654 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:26.419630 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-5grt4"] Sep 08 02:27:26.420607 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:27:26.420580 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode58e0d3e_3dca_4cb1_9020_fcfa14abd346.slice/crio-34e659f87fad33b01473949026857209ce4173a1e3f39644149bccbf50f77460 WatchSource:0}: Error finding container 34e659f87fad33b01473949026857209ce4173a1e3f39644149bccbf50f77460: Status 404 returned error can't find the container with id 34e659f87fad33b01473949026857209ce4173a1e3f39644149bccbf50f77460 Sep 08 02:27:26.650781 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:26.650755 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-q4nhg"] Sep 08 02:27:26.652986 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:27:26.652956 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c00376e_381e_43cb_8b44_c6c64323937b.slice/crio-5dab9c4a59d37f51a8c44c96d45707004d53d1b21ee6ccce58e8953acc6ca830 WatchSource:0}: Error finding container 5dab9c4a59d37f51a8c44c96d45707004d53d1b21ee6ccce58e8953acc6ca830: Status 404 returned error can't find the container with id 5dab9c4a59d37f51a8c44c96d45707004d53d1b21ee6ccce58e8953acc6ca830 Sep 08 02:27:26.712660 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:26.712628 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-q4nhg" event={"ID":"3c00376e-381e-43cb-8b44-c6c64323937b","Type":"ContainerStarted","Data":"5dab9c4a59d37f51a8c44c96d45707004d53d1b21ee6ccce58e8953acc6ca830"} Sep 08 02:27:26.713755 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:26.713731 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-5grt4" event={"ID":"e58e0d3e-3dca-4cb1-9020-fcfa14abd346","Type":"ContainerStarted","Data":"34e659f87fad33b01473949026857209ce4173a1e3f39644149bccbf50f77460"} Sep 08 02:27:28.728729 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:28.728693 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-5grt4" event={"ID":"e58e0d3e-3dca-4cb1-9020-fcfa14abd346","Type":"ContainerStarted","Data":"32a984e1f238fca63546f55a2766bfd1fb4bc928be092c2c6805b26dc1173ee1"} Sep 08 02:27:28.730162 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:28.730134 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-q4nhg" event={"ID":"3c00376e-381e-43cb-8b44-c6c64323937b","Type":"ContainerStarted","Data":"792b57a58b9725b78eca50911a165b1404877865378e4e3d3228223adcdb24ad"} Sep 08 02:27:28.746652 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:28.746598 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-5grt4" podStartSLOduration=1.633764503 podStartE2EDuration="3.746579059s" totalImagesPullingTime="2.112814556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:27:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:27:28.744823669 +0000 UTC m=+344.276468534" watchObservedRunningTime="2026-09-08 02:27:28.746579059 +0000 UTC m=+344.278223927" Sep 08 02:27:28.762140 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:28.762085 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-q4nhg" podStartSLOduration=0.869947134 podStartE2EDuration="2.76206469s" totalImagesPullingTime="1.892117556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:27:28.759946195 +0000 UTC m=+344.291591063" watchObservedRunningTime="2026-09-08 02:27:28.76206469 +0000 UTC m=+344.293709601" Sep 08 02:27:28.791906 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:28.791831 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-5grt4"] Sep 08 02:27:30.739485 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:30.739410 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-5grt4" podUID="e58e0d3e-3dca-4cb1-9020-fcfa14abd346" containerName="authorino" containerID="cri-o://32a984e1f238fca63546f55a2766bfd1fb4bc928be092c2c6805b26dc1173ee1" gracePeriod=30 Sep 08 02:27:30.980385 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:30.980353 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-5grt4" Sep 08 02:27:31.100477 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:31.100436 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5jrt\" (UniqueName: \"kubernetes.io/projected/e58e0d3e-3dca-4cb1-9020-fcfa14abd346-kube-api-access-m5jrt\") pod \"e58e0d3e-3dca-4cb1-9020-fcfa14abd346\" (UID: \"e58e0d3e-3dca-4cb1-9020-fcfa14abd346\") " Sep 08 02:27:31.102503 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:31.102480 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e58e0d3e-3dca-4cb1-9020-fcfa14abd346-kube-api-access-m5jrt" (OuterVolumeSpecName: "kube-api-access-m5jrt") pod "e58e0d3e-3dca-4cb1-9020-fcfa14abd346" (UID: "e58e0d3e-3dca-4cb1-9020-fcfa14abd346"). InnerVolumeSpecName "kube-api-access-m5jrt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 02:27:31.201679 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:31.201649 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5jrt\" (UniqueName: \"kubernetes.io/projected/e58e0d3e-3dca-4cb1-9020-fcfa14abd346-kube-api-access-m5jrt\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Sep 08 02:27:31.745811 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:31.745774 2560 generic.go:356] "Generic (PLEG): container finished" podID="e58e0d3e-3dca-4cb1-9020-fcfa14abd346" containerID="32a984e1f238fca63546f55a2766bfd1fb4bc928be092c2c6805b26dc1173ee1" exitCode=0 Sep 08 02:27:31.746238 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:31.745823 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-5grt4" Sep 08 02:27:31.746238 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:31.745856 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-5grt4" event={"ID":"e58e0d3e-3dca-4cb1-9020-fcfa14abd346","Type":"ContainerDied","Data":"32a984e1f238fca63546f55a2766bfd1fb4bc928be092c2c6805b26dc1173ee1"} Sep 08 02:27:31.746238 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:31.745893 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-5grt4" event={"ID":"e58e0d3e-3dca-4cb1-9020-fcfa14abd346","Type":"ContainerDied","Data":"34e659f87fad33b01473949026857209ce4173a1e3f39644149bccbf50f77460"} Sep 08 02:27:31.746238 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:31.745914 2560 scope.go:117] "RemoveContainer" containerID="32a984e1f238fca63546f55a2766bfd1fb4bc928be092c2c6805b26dc1173ee1" Sep 08 02:27:31.754983 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:31.754962 2560 scope.go:117] "RemoveContainer" containerID="32a984e1f238fca63546f55a2766bfd1fb4bc928be092c2c6805b26dc1173ee1" Sep 08 02:27:31.755228 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:27:31.755206 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32a984e1f238fca63546f55a2766bfd1fb4bc928be092c2c6805b26dc1173ee1\": container with ID starting with 32a984e1f238fca63546f55a2766bfd1fb4bc928be092c2c6805b26dc1173ee1 not found: ID does not exist" containerID="32a984e1f238fca63546f55a2766bfd1fb4bc928be092c2c6805b26dc1173ee1" Sep 08 02:27:31.755288 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:31.755237 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32a984e1f238fca63546f55a2766bfd1fb4bc928be092c2c6805b26dc1173ee1"} err="failed to get container status \"32a984e1f238fca63546f55a2766bfd1fb4bc928be092c2c6805b26dc1173ee1\": rpc error: code = NotFound desc = could not find container \"32a984e1f238fca63546f55a2766bfd1fb4bc928be092c2c6805b26dc1173ee1\": container with ID starting with 32a984e1f238fca63546f55a2766bfd1fb4bc928be092c2c6805b26dc1173ee1 not found: ID does not exist" Sep 08 02:27:31.767975 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:31.767950 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-5grt4"] Sep 08 02:27:31.772276 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:31.772255 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-5grt4"] Sep 08 02:27:33.017352 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:33.017314 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e58e0d3e-3dca-4cb1-9020-fcfa14abd346" path="/var/lib/kubelet/pods/e58e0d3e-3dca-4cb1-9020-fcfa14abd346/volumes" Sep 08 02:27:33.698654 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:33.698621 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-657f59599b-dpqc5" Sep 08 02:27:55.635968 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.635612 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-768897d8b6-82v2h"] Sep 08 02:27:55.637100 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.637067 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e58e0d3e-3dca-4cb1-9020-fcfa14abd346" containerName="authorino" Sep 08 02:27:55.637216 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.637205 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e58e0d3e-3dca-4cb1-9020-fcfa14abd346" containerName="authorino" Sep 08 02:27:55.637385 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.637364 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e58e0d3e-3dca-4cb1-9020-fcfa14abd346" containerName="authorino" Sep 08 02:27:55.649748 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.649636 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" Sep 08 02:27:55.653203 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.653182 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 08 02:27:55.653680 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.653365 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-768897d8b6-82v2h"] Sep 08 02:27:55.653983 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.653919 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 08 02:27:55.658877 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.658048 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 08 02:27:55.659138 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.658123 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7gn54\"" Sep 08 02:27:55.659350 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.658165 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 02:27:55.694627 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.691417 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-85d49875c6-xnn5f"] Sep 08 02:27:55.695726 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.695705 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-85d49875c6-xnn5f" Sep 08 02:27:55.702482 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.701897 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-8xdh5\"" Sep 08 02:27:55.707481 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.703088 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Sep 08 02:27:55.707481 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.705994 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-85d49875c6-xnn5f"] Sep 08 02:27:55.725965 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.725937 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrmp7\" (UniqueName: \"kubernetes.io/projected/6f25feac-0778-47fd-beac-dae58cbbb2ea-kube-api-access-wrmp7\") pod \"maas-api-768897d8b6-82v2h\" (UID: \"6f25feac-0778-47fd-beac-dae58cbbb2ea\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" Sep 08 02:27:55.726102 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.725991 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f25feac-0778-47fd-beac-dae58cbbb2ea-metrics-certs\") pod \"maas-api-768897d8b6-82v2h\" (UID: \"6f25feac-0778-47fd-beac-dae58cbbb2ea\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" Sep 08 02:27:55.726102 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.726074 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f25feac-0778-47fd-beac-dae58cbbb2ea-maas-api-tls\") pod \"maas-api-768897d8b6-82v2h\" (UID: \"6f25feac-0778-47fd-beac-dae58cbbb2ea\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" Sep 08 02:27:55.726220 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.726100 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f25feac-0778-47fd-beac-dae58cbbb2ea-openshift-service-ca\") pod \"maas-api-768897d8b6-82v2h\" (UID: \"6f25feac-0778-47fd-beac-dae58cbbb2ea\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" Sep 08 02:27:55.764484 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.760481 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-7d6487fcc8-gqr6x"] Sep 08 02:27:55.764965 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.764948 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-7d6487fcc8-gqr6x" Sep 08 02:27:55.769289 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.769269 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Sep 08 02:27:55.769711 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.769693 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-czhmn\"" Sep 08 02:27:55.778498 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.778477 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-7d6487fcc8-gqr6x"] Sep 08 02:27:55.797579 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.797531 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-bb7b75cc4-g7dxh"] Sep 08 02:27:55.802413 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.802395 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-bb7b75cc4-g7dxh" Sep 08 02:27:55.806262 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.806240 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Sep 08 02:27:55.808157 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.807035 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-hdxqj\"" Sep 08 02:27:55.808157 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.807210 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\"" Sep 08 02:27:55.816750 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.816367 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-bb7b75cc4-g7dxh"] Sep 08 02:27:55.826875 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.826855 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f25feac-0778-47fd-beac-dae58cbbb2ea-maas-api-tls\") pod \"maas-api-768897d8b6-82v2h\" (UID: \"6f25feac-0778-47fd-beac-dae58cbbb2ea\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" Sep 08 02:27:55.828072 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.828052 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f25feac-0778-47fd-beac-dae58cbbb2ea-openshift-service-ca\") pod \"maas-api-768897d8b6-82v2h\" (UID: \"6f25feac-0778-47fd-beac-dae58cbbb2ea\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" Sep 08 02:27:55.829011 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.828995 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrmp7\" (UniqueName: \"kubernetes.io/projected/6f25feac-0778-47fd-beac-dae58cbbb2ea-kube-api-access-wrmp7\") pod \"maas-api-768897d8b6-82v2h\" (UID: \"6f25feac-0778-47fd-beac-dae58cbbb2ea\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" Sep 08 02:27:55.829429 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.829410 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c45c3758-5159-43b1-a8b0-58c14735ae03-cert\") pod \"kserve-controller-manager-85d49875c6-xnn5f\" (UID: \"c45c3758-5159-43b1-a8b0-58c14735ae03\") " pod="opendatahub/kserve-controller-manager-85d49875c6-xnn5f" Sep 08 02:27:55.829593 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.829577 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f25feac-0778-47fd-beac-dae58cbbb2ea-metrics-certs\") pod \"maas-api-768897d8b6-82v2h\" (UID: \"6f25feac-0778-47fd-beac-dae58cbbb2ea\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" Sep 08 02:27:55.829753 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.829737 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdd8s\" (UniqueName: \"kubernetes.io/projected/2c57646e-c736-4ca0-9f9a-8c271a1e8169-kube-api-access-gdd8s\") pod \"model-serving-api-7d6487fcc8-gqr6x\" (UID: \"2c57646e-c736-4ca0-9f9a-8c271a1e8169\") " pod="opendatahub/model-serving-api-7d6487fcc8-gqr6x" Sep 08 02:27:55.829866 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.829853 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c57646e-c736-4ca0-9f9a-8c271a1e8169-tls-certs\") pod \"model-serving-api-7d6487fcc8-gqr6x\" (UID: \"2c57646e-c736-4ca0-9f9a-8c271a1e8169\") " pod="opendatahub/model-serving-api-7d6487fcc8-gqr6x" Sep 08 02:27:55.829960 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.829947 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9cg7\" (UniqueName: \"kubernetes.io/projected/c45c3758-5159-43b1-a8b0-58c14735ae03-kube-api-access-h9cg7\") pod \"kserve-controller-manager-85d49875c6-xnn5f\" (UID: \"c45c3758-5159-43b1-a8b0-58c14735ae03\") " pod="opendatahub/kserve-controller-manager-85d49875c6-xnn5f" Sep 08 02:27:55.830079 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:27:55.828005 2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Sep 08 02:27:55.830190 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:27:55.830181 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f25feac-0778-47fd-beac-dae58cbbb2ea-maas-api-tls podName:6f25feac-0778-47fd-beac-dae58cbbb2ea nodeName:}" failed. No retries permitted until 2026-09-08 02:27:56.330163106 +0000 UTC m=+371.861807954 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6f25feac-0778-47fd-beac-dae58cbbb2ea-maas-api-tls") pod "maas-api-768897d8b6-82v2h" (UID: "6f25feac-0778-47fd-beac-dae58cbbb2ea") : secret "maas-api-serving-cert" not found Sep 08 02:27:55.830434 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.828913 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f25feac-0778-47fd-beac-dae58cbbb2ea-openshift-service-ca\") pod \"maas-api-768897d8b6-82v2h\" (UID: \"6f25feac-0778-47fd-beac-dae58cbbb2ea\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" Sep 08 02:27:55.830622 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:27:55.830610 2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found Sep 08 02:27:55.830728 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:27:55.830719 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f25feac-0778-47fd-beac-dae58cbbb2ea-metrics-certs podName:6f25feac-0778-47fd-beac-dae58cbbb2ea nodeName:}" failed. No retries permitted until 2026-09-08 02:27:56.330705825 +0000 UTC m=+371.862350674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f25feac-0778-47fd-beac-dae58cbbb2ea-metrics-certs") pod "maas-api-768897d8b6-82v2h" (UID: "6f25feac-0778-47fd-beac-dae58cbbb2ea") : secret "maas-api-metrics-tls" not found Sep 08 02:27:55.846038 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.845996 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrmp7\" (UniqueName: \"kubernetes.io/projected/6f25feac-0778-47fd-beac-dae58cbbb2ea-kube-api-access-wrmp7\") pod \"maas-api-768897d8b6-82v2h\" (UID: \"6f25feac-0778-47fd-beac-dae58cbbb2ea\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" Sep 08 02:27:55.930793 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.930707 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ba0375c-8d15-4d23-b19e-63c45bfa0e2a-cert\") pod \"odh-model-controller-bb7b75cc4-g7dxh\" (UID: \"6ba0375c-8d15-4d23-b19e-63c45bfa0e2a\") " pod="opendatahub/odh-model-controller-bb7b75cc4-g7dxh" Sep 08 02:27:55.930793 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.930766 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c45c3758-5159-43b1-a8b0-58c14735ae03-cert\") pod \"kserve-controller-manager-85d49875c6-xnn5f\" (UID: \"c45c3758-5159-43b1-a8b0-58c14735ae03\") " pod="opendatahub/kserve-controller-manager-85d49875c6-xnn5f" Sep 08 02:27:55.931007 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.930874 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdd8s\" (UniqueName: \"kubernetes.io/projected/2c57646e-c736-4ca0-9f9a-8c271a1e8169-kube-api-access-gdd8s\") pod \"model-serving-api-7d6487fcc8-gqr6x\" (UID: \"2c57646e-c736-4ca0-9f9a-8c271a1e8169\") " pod="opendatahub/model-serving-api-7d6487fcc8-gqr6x" Sep 08 02:27:55.931007 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.930913 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ba0375c-8d15-4d23-b19e-63c45bfa0e2a-metrics-certs\") pod \"odh-model-controller-bb7b75cc4-g7dxh\" (UID: \"6ba0375c-8d15-4d23-b19e-63c45bfa0e2a\") " pod="opendatahub/odh-model-controller-bb7b75cc4-g7dxh" Sep 08 02:27:55.931007 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.930940 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c57646e-c736-4ca0-9f9a-8c271a1e8169-tls-certs\") pod \"model-serving-api-7d6487fcc8-gqr6x\" (UID: \"2c57646e-c736-4ca0-9f9a-8c271a1e8169\") " pod="opendatahub/model-serving-api-7d6487fcc8-gqr6x" Sep 08 02:27:55.931007 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.930961 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2wkp\" (UniqueName: \"kubernetes.io/projected/6ba0375c-8d15-4d23-b19e-63c45bfa0e2a-kube-api-access-x2wkp\") pod \"odh-model-controller-bb7b75cc4-g7dxh\" (UID: \"6ba0375c-8d15-4d23-b19e-63c45bfa0e2a\") " pod="opendatahub/odh-model-controller-bb7b75cc4-g7dxh" Sep 08 02:27:55.931007 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.930994 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9cg7\" (UniqueName: \"kubernetes.io/projected/c45c3758-5159-43b1-a8b0-58c14735ae03-kube-api-access-h9cg7\") pod \"kserve-controller-manager-85d49875c6-xnn5f\" (UID: \"c45c3758-5159-43b1-a8b0-58c14735ae03\") " pod="opendatahub/kserve-controller-manager-85d49875c6-xnn5f" Sep 08 02:27:55.931681 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:27:55.931429 2560 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Sep 08 02:27:55.931681 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:27:55.931516 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c57646e-c736-4ca0-9f9a-8c271a1e8169-tls-certs podName:2c57646e-c736-4ca0-9f9a-8c271a1e8169 nodeName:}" failed. No retries permitted until 2026-09-08 02:27:56.431496829 +0000 UTC m=+371.963141687 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2c57646e-c736-4ca0-9f9a-8c271a1e8169-tls-certs") pod "model-serving-api-7d6487fcc8-gqr6x" (UID: "2c57646e-c736-4ca0-9f9a-8c271a1e8169") : secret "model-serving-api-tls" not found Sep 08 02:27:55.931681 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:27:55.931619 2560 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Sep 08 02:27:55.931681 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:27:55.931662 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c45c3758-5159-43b1-a8b0-58c14735ae03-cert podName:c45c3758-5159-43b1-a8b0-58c14735ae03 nodeName:}" failed. No retries permitted until 2026-09-08 02:27:56.431649844 +0000 UTC m=+371.963294688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c45c3758-5159-43b1-a8b0-58c14735ae03-cert") pod "kserve-controller-manager-85d49875c6-xnn5f" (UID: "c45c3758-5159-43b1-a8b0-58c14735ae03") : secret "kserve-webhook-server-cert" not found Sep 08 02:27:55.948479 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.947836 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdd8s\" (UniqueName: \"kubernetes.io/projected/2c57646e-c736-4ca0-9f9a-8c271a1e8169-kube-api-access-gdd8s\") pod \"model-serving-api-7d6487fcc8-gqr6x\" (UID: \"2c57646e-c736-4ca0-9f9a-8c271a1e8169\") " pod="opendatahub/model-serving-api-7d6487fcc8-gqr6x" Sep 08 02:27:55.948479 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:55.948475 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9cg7\" (UniqueName: \"kubernetes.io/projected/c45c3758-5159-43b1-a8b0-58c14735ae03-kube-api-access-h9cg7\") pod \"kserve-controller-manager-85d49875c6-xnn5f\" (UID: \"c45c3758-5159-43b1-a8b0-58c14735ae03\") " pod="opendatahub/kserve-controller-manager-85d49875c6-xnn5f" Sep 08 02:27:56.033348 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.031789 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ba0375c-8d15-4d23-b19e-63c45bfa0e2a-metrics-certs\") pod \"odh-model-controller-bb7b75cc4-g7dxh\" (UID: \"6ba0375c-8d15-4d23-b19e-63c45bfa0e2a\") " pod="opendatahub/odh-model-controller-bb7b75cc4-g7dxh" Sep 08 02:27:56.033348 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.031846 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2wkp\" (UniqueName: \"kubernetes.io/projected/6ba0375c-8d15-4d23-b19e-63c45bfa0e2a-kube-api-access-x2wkp\") pod \"odh-model-controller-bb7b75cc4-g7dxh\" (UID: \"6ba0375c-8d15-4d23-b19e-63c45bfa0e2a\") " pod="opendatahub/odh-model-controller-bb7b75cc4-g7dxh" Sep 08 02:27:56.033348 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.031910 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ba0375c-8d15-4d23-b19e-63c45bfa0e2a-cert\") pod \"odh-model-controller-bb7b75cc4-g7dxh\" (UID: \"6ba0375c-8d15-4d23-b19e-63c45bfa0e2a\") " pod="opendatahub/odh-model-controller-bb7b75cc4-g7dxh" Sep 08 02:27:56.033348 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:27:56.032054 2560 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Sep 08 02:27:56.033348 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:27:56.032107 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ba0375c-8d15-4d23-b19e-63c45bfa0e2a-cert podName:6ba0375c-8d15-4d23-b19e-63c45bfa0e2a nodeName:}" failed. No retries permitted until 2026-09-08 02:27:56.532089657 +0000 UTC m=+372.063734507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6ba0375c-8d15-4d23-b19e-63c45bfa0e2a-cert") pod "odh-model-controller-bb7b75cc4-g7dxh" (UID: "6ba0375c-8d15-4d23-b19e-63c45bfa0e2a") : secret "odh-model-controller-webhook-cert" not found Sep 08 02:27:56.033348 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:27:56.032225 2560 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-metrics-tls: secret "odh-model-controller-metrics-tls" not found Sep 08 02:27:56.033348 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:27:56.032291 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ba0375c-8d15-4d23-b19e-63c45bfa0e2a-metrics-certs podName:6ba0375c-8d15-4d23-b19e-63c45bfa0e2a nodeName:}" failed. No retries permitted until 2026-09-08 02:27:56.532275368 +0000 UTC m=+372.063920216 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ba0375c-8d15-4d23-b19e-63c45bfa0e2a-metrics-certs") pod "odh-model-controller-bb7b75cc4-g7dxh" (UID: "6ba0375c-8d15-4d23-b19e-63c45bfa0e2a") : secret "odh-model-controller-metrics-tls" not found Sep 08 02:27:56.042293 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.042270 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2wkp\" (UniqueName: \"kubernetes.io/projected/6ba0375c-8d15-4d23-b19e-63c45bfa0e2a-kube-api-access-x2wkp\") pod \"odh-model-controller-bb7b75cc4-g7dxh\" (UID: \"6ba0375c-8d15-4d23-b19e-63c45bfa0e2a\") " pod="opendatahub/odh-model-controller-bb7b75cc4-g7dxh" Sep 08 02:27:56.334999 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.334912 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f25feac-0778-47fd-beac-dae58cbbb2ea-metrics-certs\") pod \"maas-api-768897d8b6-82v2h\" (UID: \"6f25feac-0778-47fd-beac-dae58cbbb2ea\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" Sep 08 02:27:56.335155 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.335050 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f25feac-0778-47fd-beac-dae58cbbb2ea-maas-api-tls\") pod \"maas-api-768897d8b6-82v2h\" (UID: \"6f25feac-0778-47fd-beac-dae58cbbb2ea\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" Sep 08 02:27:56.337334 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.337306 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f25feac-0778-47fd-beac-dae58cbbb2ea-metrics-certs\") pod \"maas-api-768897d8b6-82v2h\" (UID: \"6f25feac-0778-47fd-beac-dae58cbbb2ea\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" Sep 08 02:27:56.337481 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.337378 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f25feac-0778-47fd-beac-dae58cbbb2ea-maas-api-tls\") pod \"maas-api-768897d8b6-82v2h\" (UID: \"6f25feac-0778-47fd-beac-dae58cbbb2ea\") " pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" Sep 08 02:27:56.436243 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.436209 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c57646e-c736-4ca0-9f9a-8c271a1e8169-tls-certs\") pod \"model-serving-api-7d6487fcc8-gqr6x\" (UID: \"2c57646e-c736-4ca0-9f9a-8c271a1e8169\") " pod="opendatahub/model-serving-api-7d6487fcc8-gqr6x" Sep 08 02:27:56.436426 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.436303 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c45c3758-5159-43b1-a8b0-58c14735ae03-cert\") pod \"kserve-controller-manager-85d49875c6-xnn5f\" (UID: \"c45c3758-5159-43b1-a8b0-58c14735ae03\") " pod="opendatahub/kserve-controller-manager-85d49875c6-xnn5f" Sep 08 02:27:56.439136 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.439110 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c45c3758-5159-43b1-a8b0-58c14735ae03-cert\") pod \"kserve-controller-manager-85d49875c6-xnn5f\" (UID: \"c45c3758-5159-43b1-a8b0-58c14735ae03\") " pod="opendatahub/kserve-controller-manager-85d49875c6-xnn5f" Sep 08 02:27:56.439255 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.439110 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c57646e-c736-4ca0-9f9a-8c271a1e8169-tls-certs\") pod \"model-serving-api-7d6487fcc8-gqr6x\" (UID: \"2c57646e-c736-4ca0-9f9a-8c271a1e8169\") " pod="opendatahub/model-serving-api-7d6487fcc8-gqr6x" Sep 08 02:27:56.537511 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.537477 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ba0375c-8d15-4d23-b19e-63c45bfa0e2a-metrics-certs\") pod \"odh-model-controller-bb7b75cc4-g7dxh\" (UID: \"6ba0375c-8d15-4d23-b19e-63c45bfa0e2a\") " pod="opendatahub/odh-model-controller-bb7b75cc4-g7dxh" Sep 08 02:27:56.537670 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.537549 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ba0375c-8d15-4d23-b19e-63c45bfa0e2a-cert\") pod \"odh-model-controller-bb7b75cc4-g7dxh\" (UID: \"6ba0375c-8d15-4d23-b19e-63c45bfa0e2a\") " pod="opendatahub/odh-model-controller-bb7b75cc4-g7dxh" Sep 08 02:27:56.537718 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:27:56.537696 2560 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Sep 08 02:27:56.537796 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:27:56.537784 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ba0375c-8d15-4d23-b19e-63c45bfa0e2a-cert podName:6ba0375c-8d15-4d23-b19e-63c45bfa0e2a nodeName:}" failed. No retries permitted until 2026-09-08 02:27:57.537763572 +0000 UTC m=+373.069408422 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6ba0375c-8d15-4d23-b19e-63c45bfa0e2a-cert") pod "odh-model-controller-bb7b75cc4-g7dxh" (UID: "6ba0375c-8d15-4d23-b19e-63c45bfa0e2a") : secret "odh-model-controller-webhook-cert" not found Sep 08 02:27:56.539886 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.539862 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ba0375c-8d15-4d23-b19e-63c45bfa0e2a-metrics-certs\") pod \"odh-model-controller-bb7b75cc4-g7dxh\" (UID: \"6ba0375c-8d15-4d23-b19e-63c45bfa0e2a\") " pod="opendatahub/odh-model-controller-bb7b75cc4-g7dxh" Sep 08 02:27:56.547786 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.547764 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-8jtjc"] Sep 08 02:27:56.552833 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.552814 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-8jtjc" Sep 08 02:27:56.555653 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.555629 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 08 02:27:56.555919 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.555903 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-84lg2\"" Sep 08 02:27:56.564770 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.564749 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-8jtjc"] Sep 08 02:27:56.600872 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.600849 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" Sep 08 02:27:56.616129 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.616100 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-85d49875c6-xnn5f" Sep 08 02:27:56.639028 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.638971 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eedd08c8-e8e8-4464-9da0-c522a2d09a2b-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-8jtjc\" (UID: \"eedd08c8-e8e8-4464-9da0-c522a2d09a2b\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8jtjc" Sep 08 02:27:56.639422 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.639153 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gh4q\" (UniqueName: \"kubernetes.io/projected/eedd08c8-e8e8-4464-9da0-c522a2d09a2b-kube-api-access-5gh4q\") pod \"payload-processing-5b54b7fbb7-8jtjc\" (UID: \"eedd08c8-e8e8-4464-9da0-c522a2d09a2b\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8jtjc" Sep 08 02:27:56.678181 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.677876 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-7d6487fcc8-gqr6x" Sep 08 02:27:56.720355 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.720311 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv"] Sep 08 02:27:56.740808 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.740527 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gh4q\" (UniqueName: \"kubernetes.io/projected/eedd08c8-e8e8-4464-9da0-c522a2d09a2b-kube-api-access-5gh4q\") pod \"payload-processing-5b54b7fbb7-8jtjc\" (UID: \"eedd08c8-e8e8-4464-9da0-c522a2d09a2b\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8jtjc" Sep 08 02:27:56.740808 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.740642 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eedd08c8-e8e8-4464-9da0-c522a2d09a2b-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-8jtjc\" (UID: \"eedd08c8-e8e8-4464-9da0-c522a2d09a2b\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8jtjc" Sep 08 02:27:56.741985 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.741954 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eedd08c8-e8e8-4464-9da0-c522a2d09a2b-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-8jtjc\" (UID: \"eedd08c8-e8e8-4464-9da0-c522a2d09a2b\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8jtjc" Sep 08 02:27:56.751573 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.751498 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gh4q\" (UniqueName: \"kubernetes.io/projected/eedd08c8-e8e8-4464-9da0-c522a2d09a2b-kube-api-access-5gh4q\") pod \"payload-processing-5b54b7fbb7-8jtjc\" (UID: \"eedd08c8-e8e8-4464-9da0-c522a2d09a2b\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-8jtjc" Sep 08 02:27:56.865228 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.865110 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-8jtjc" Sep 08 02:27:56.886248 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.886169 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-768897d8b6-82v2h"] Sep 08 02:27:56.889060 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:27:56.888984 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f25feac_0778_47fd_beac_dae58cbbb2ea.slice/crio-64f424b1fac0aa145bd4be70fd67e42f26a886fda16ade39406c84684f4a9e61 WatchSource:0}: Error finding container 64f424b1fac0aa145bd4be70fd67e42f26a886fda16ade39406c84684f4a9e61: Status 404 returned error can't find the container with id 64f424b1fac0aa145bd4be70fd67e42f26a886fda16ade39406c84684f4a9e61 Sep 08 02:27:56.920353 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.920152 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-85d49875c6-xnn5f"] Sep 08 02:27:56.925509 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:27:56.924186 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc45c3758_5159_43b1_a8b0_58c14735ae03.slice/crio-602cf13f1281aae763872a5b47f6b2cfba86ed6b731a119cd19966dee46a324b WatchSource:0}: Error finding container 602cf13f1281aae763872a5b47f6b2cfba86ed6b731a119cd19966dee46a324b: Status 404 returned error can't find the container with id 602cf13f1281aae763872a5b47f6b2cfba86ed6b731a119cd19966dee46a324b Sep 08 02:27:56.986953 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:56.986903 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-7d6487fcc8-gqr6x"] Sep 08 02:27:56.989830 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:27:56.989790 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c57646e_c736_4ca0_9f9a_8c271a1e8169.slice/crio-c5bac7ec64c0ea749b536f4c6cb679cbb0ac63c620a409783d5ebe63e4a17599 WatchSource:0}: Error finding container c5bac7ec64c0ea749b536f4c6cb679cbb0ac63c620a409783d5ebe63e4a17599: Status 404 returned error can't find the container with id c5bac7ec64c0ea749b536f4c6cb679cbb0ac63c620a409783d5ebe63e4a17599 Sep 08 02:27:57.156306 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:57.156266 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-8jtjc"] Sep 08 02:27:57.157643 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:27:57.157622 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeedd08c8_e8e8_4464_9da0_c522a2d09a2b.slice/crio-ea04c12cdb8c2234eb44267189b85155793366406a5b12e1a2e4119421b31457 WatchSource:0}: Error finding container ea04c12cdb8c2234eb44267189b85155793366406a5b12e1a2e4119421b31457: Status 404 returned error can't find the container with id ea04c12cdb8c2234eb44267189b85155793366406a5b12e1a2e4119421b31457 Sep 08 02:27:57.549637 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:57.549392 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ba0375c-8d15-4d23-b19e-63c45bfa0e2a-cert\") pod \"odh-model-controller-bb7b75cc4-g7dxh\" (UID: \"6ba0375c-8d15-4d23-b19e-63c45bfa0e2a\") " pod="opendatahub/odh-model-controller-bb7b75cc4-g7dxh" Sep 08 02:27:57.552745 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:57.552671 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ba0375c-8d15-4d23-b19e-63c45bfa0e2a-cert\") pod \"odh-model-controller-bb7b75cc4-g7dxh\" (UID: \"6ba0375c-8d15-4d23-b19e-63c45bfa0e2a\") " pod="opendatahub/odh-model-controller-bb7b75cc4-g7dxh" Sep 08 02:27:57.619825 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:57.619792 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-bb7b75cc4-g7dxh" Sep 08 02:27:57.889814 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:57.889743 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-7d6487fcc8-gqr6x" event={"ID":"2c57646e-c736-4ca0-9f9a-8c271a1e8169","Type":"ContainerStarted","Data":"c5bac7ec64c0ea749b536f4c6cb679cbb0ac63c620a409783d5ebe63e4a17599"} Sep 08 02:27:57.893385 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:57.893315 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-8jtjc" event={"ID":"eedd08c8-e8e8-4464-9da0-c522a2d09a2b","Type":"ContainerStarted","Data":"ea04c12cdb8c2234eb44267189b85155793366406a5b12e1a2e4119421b31457"} Sep 08 02:27:57.897611 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:57.897554 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" event={"ID":"6f25feac-0778-47fd-beac-dae58cbbb2ea","Type":"ContainerStarted","Data":"64f424b1fac0aa145bd4be70fd67e42f26a886fda16ade39406c84684f4a9e61"} Sep 08 02:27:57.899570 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:57.899513 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-85d49875c6-xnn5f" event={"ID":"c45c3758-5159-43b1-a8b0-58c14735ae03","Type":"ContainerStarted","Data":"602cf13f1281aae763872a5b47f6b2cfba86ed6b731a119cd19966dee46a324b"} Sep 08 02:27:57.923061 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:57.922851 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-bb7b75cc4-g7dxh"] Sep 08 02:27:58.909413 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:27:58.909326 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-bb7b75cc4-g7dxh" event={"ID":"6ba0375c-8d15-4d23-b19e-63c45bfa0e2a","Type":"ContainerStarted","Data":"ceda6dfa9496f763100610b9324d523fd39e9562ab3747d440fad52e01668af3"} Sep 08 02:28:02.934713 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:02.934676 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-8jtjc" event={"ID":"eedd08c8-e8e8-4464-9da0-c522a2d09a2b","Type":"ContainerStarted","Data":"3d431753a9c634c1f0953d3a0fe306fc0ead4065731bcab13568a8b4c3ccb16d"} Sep 08 02:28:02.935197 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:02.934867 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-8jtjc" Sep 08 02:28:02.936338 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:02.936311 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" event={"ID":"6f25feac-0778-47fd-beac-dae58cbbb2ea","Type":"ContainerStarted","Data":"aee66e4604267ab0f9d67266fb5f0cec45e7b304c2a269213c040ad564fa679d"} Sep 08 02:28:02.936491 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:02.936370 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" Sep 08 02:28:02.937956 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:02.937930 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-85d49875c6-xnn5f" event={"ID":"c45c3758-5159-43b1-a8b0-58c14735ae03","Type":"ContainerStarted","Data":"a6232d3875d422ea56eb095589adedf77fcfbfed64639f1b847fa2e439e76292"} Sep 08 02:28:02.938072 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:02.938019 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-85d49875c6-xnn5f" Sep 08 02:28:02.939601 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:02.939574 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-bb7b75cc4-g7dxh" event={"ID":"6ba0375c-8d15-4d23-b19e-63c45bfa0e2a","Type":"ContainerStarted","Data":"0d78ddf89599cfc9242c8230fa83b96c22770cf534aa402037c1dfd0eb29f80c"} Sep 08 02:28:02.939726 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:02.939649 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-bb7b75cc4-g7dxh" Sep 08 02:28:02.941108 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:02.941085 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-7d6487fcc8-gqr6x" event={"ID":"2c57646e-c736-4ca0-9f9a-8c271a1e8169","Type":"ContainerStarted","Data":"5484bb915e7fca74dca48792cb269bbf580b56a3d2d04add33f1180e32db77c0"} Sep 08 02:28:02.941230 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:02.941203 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-7d6487fcc8-gqr6x" Sep 08 02:28:02.958680 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:02.958633 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-8jtjc" podStartSLOduration=2.308419905 podStartE2EDuration="6.958617688s" totalImagesPullingTime="4.650197783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:27:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:28:02.956565921 +0000 UTC m=+378.488210780" watchObservedRunningTime="2026-09-08 02:28:02.958617688 +0000 UTC m=+378.490262555" Sep 08 02:28:03.016144 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:03.016097 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-85d49875c6-xnn5f" podStartSLOduration=3.128353375 podStartE2EDuration="8.016080301s" totalImagesPullingTime="4.887726926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:27:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:28:03.015275298 +0000 UTC m=+378.546920163" watchObservedRunningTime="2026-09-08 02:28:03.016080301 +0000 UTC m=+378.547725159" Sep 08 02:28:03.016763 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:03.016717 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-bb7b75cc4-g7dxh" podStartSLOduration=4.121671543 podStartE2EDuration="8.016706879s" totalImagesPullingTime="3.895035336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:27:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:28:02.988157374 +0000 UTC m=+378.519802238" watchObservedRunningTime="2026-09-08 02:28:03.016706879 +0000 UTC m=+378.548351738" Sep 08 02:28:03.047145 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:03.047093 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" podStartSLOduration=3.127703866 podStartE2EDuration="8.047072503s" totalImagesPullingTime="4.919368637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:27:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:28:03.044035475 +0000 UTC m=+378.575680341" watchObservedRunningTime="2026-09-08 02:28:03.047072503 +0000 UTC m=+378.578717369" Sep 08 02:28:08.952937 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:08.952908 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" Sep 08 02:28:08.974754 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:08.974711 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-7d6487fcc8-gqr6x" podStartSLOduration=9.161171376 podStartE2EDuration="13.974696144s" totalImagesPullingTime="4.813524768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:27:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:28:03.065317247 +0000 UTC m=+378.596962114" watchObservedRunningTime="2026-09-08 02:28:08.974696144 +0000 UTC m=+384.506341005" Sep 08 02:28:09.494208 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:09.494171 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-rf9sq"] Sep 08 02:28:09.497853 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:09.497832 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-rf9sq" Sep 08 02:28:09.506484 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:09.506440 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-rf9sq"] Sep 08 02:28:09.576854 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:09.576818 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frbq5\" (UniqueName: \"kubernetes.io/projected/dcbe2c77-26d2-46af-905a-5aecc5face16-kube-api-access-frbq5\") pod \"authorino-5dc7ccb5f-rf9sq\" (UID: \"dcbe2c77-26d2-46af-905a-5aecc5face16\") " pod="kuadrant-system/authorino-5dc7ccb5f-rf9sq" Sep 08 02:28:09.677300 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:09.677267 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frbq5\" (UniqueName: \"kubernetes.io/projected/dcbe2c77-26d2-46af-905a-5aecc5face16-kube-api-access-frbq5\") pod \"authorino-5dc7ccb5f-rf9sq\" (UID: \"dcbe2c77-26d2-46af-905a-5aecc5face16\") " pod="kuadrant-system/authorino-5dc7ccb5f-rf9sq" Sep 08 02:28:09.686852 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:09.686820 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frbq5\" (UniqueName: \"kubernetes.io/projected/dcbe2c77-26d2-46af-905a-5aecc5face16-kube-api-access-frbq5\") pod \"authorino-5dc7ccb5f-rf9sq\" (UID: \"dcbe2c77-26d2-46af-905a-5aecc5face16\") " pod="kuadrant-system/authorino-5dc7ccb5f-rf9sq" Sep 08 02:28:09.734567 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:09.734534 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-rf9sq"] Sep 08 02:28:09.734817 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:09.734804 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-rf9sq" Sep 08 02:28:09.764678 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:09.764650 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-66b58bc974-fjz4g"] Sep 08 02:28:09.768377 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:09.768357 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66b58bc974-fjz4g" Sep 08 02:28:09.776675 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:09.776621 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66b58bc974-fjz4g"] Sep 08 02:28:09.778735 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:09.778707 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t4cz\" (UniqueName: \"kubernetes.io/projected/2783a172-49ac-4664-8ef8-083b08e7416a-kube-api-access-8t4cz\") pod \"authorino-66b58bc974-fjz4g\" (UID: \"2783a172-49ac-4664-8ef8-083b08e7416a\") " pod="kuadrant-system/authorino-66b58bc974-fjz4g" Sep 08 02:28:09.880373 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:09.880340 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t4cz\" (UniqueName: \"kubernetes.io/projected/2783a172-49ac-4664-8ef8-083b08e7416a-kube-api-access-8t4cz\") pod \"authorino-66b58bc974-fjz4g\" (UID: \"2783a172-49ac-4664-8ef8-083b08e7416a\") " pod="kuadrant-system/authorino-66b58bc974-fjz4g" Sep 08 02:28:09.889370 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:09.889336 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t4cz\" (UniqueName: \"kubernetes.io/projected/2783a172-49ac-4664-8ef8-083b08e7416a-kube-api-access-8t4cz\") pod \"authorino-66b58bc974-fjz4g\" (UID: \"2783a172-49ac-4664-8ef8-083b08e7416a\") " pod="kuadrant-system/authorino-66b58bc974-fjz4g" Sep 08 02:28:10.013195 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:10.013169 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-rf9sq"] Sep 08 02:28:10.014976 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:28:10.014948 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddcbe2c77_26d2_46af_905a_5aecc5face16.slice/crio-304caddf56dd017d57001d8baa7cbccd360cf0da7997727de23d37a709e3810a WatchSource:0}: Error finding container 304caddf56dd017d57001d8baa7cbccd360cf0da7997727de23d37a709e3810a: Status 404 returned error can't find the container with id 304caddf56dd017d57001d8baa7cbccd360cf0da7997727de23d37a709e3810a Sep 08 02:28:10.082501 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:10.082451 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66b58bc974-fjz4g" Sep 08 02:28:10.354938 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:10.354909 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66b58bc974-fjz4g"] Sep 08 02:28:10.356585 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:28:10.356562 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2783a172_49ac_4664_8ef8_083b08e7416a.slice/crio-310650c46dc31c0c35e369f45cd1df678f9a5bde3a2cc1713ad8f644f91b7652 WatchSource:0}: Error finding container 310650c46dc31c0c35e369f45cd1df678f9a5bde3a2cc1713ad8f644f91b7652: Status 404 returned error can't find the container with id 310650c46dc31c0c35e369f45cd1df678f9a5bde3a2cc1713ad8f644f91b7652 Sep 08 02:28:10.987609 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:10.987570 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-rf9sq" event={"ID":"dcbe2c77-26d2-46af-905a-5aecc5face16","Type":"ContainerStarted","Data":"794b29dae200842f0b00667903f024198350c754bfb2e48c70be6bc84497500d"} Sep 08 02:28:10.987789 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:10.987623 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-rf9sq" event={"ID":"dcbe2c77-26d2-46af-905a-5aecc5face16","Type":"ContainerStarted","Data":"304caddf56dd017d57001d8baa7cbccd360cf0da7997727de23d37a709e3810a"} Sep 08 02:28:10.987789 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:10.987635 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5dc7ccb5f-rf9sq" podUID="dcbe2c77-26d2-46af-905a-5aecc5face16" containerName="authorino" containerID="cri-o://794b29dae200842f0b00667903f024198350c754bfb2e48c70be6bc84497500d" gracePeriod=30 Sep 08 02:28:10.990189 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:10.990159 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66b58bc974-fjz4g" event={"ID":"2783a172-49ac-4664-8ef8-083b08e7416a","Type":"ContainerStarted","Data":"98fdccb064eb0eeed9acf0c453ec4ac1c4804b16e4dd8e2babde7a3e676f45a0"} Sep 08 02:28:10.990296 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:10.990202 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66b58bc974-fjz4g" event={"ID":"2783a172-49ac-4664-8ef8-083b08e7416a","Type":"ContainerStarted","Data":"310650c46dc31c0c35e369f45cd1df678f9a5bde3a2cc1713ad8f644f91b7652"} Sep 08 02:28:11.008004 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.007944 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5dc7ccb5f-rf9sq" podStartSLOduration=1.6193138459999998 podStartE2EDuration="2.007923491s" totalImagesPullingTime="388.609645ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:28:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:28:11.007604028 +0000 UTC m=+386.539248894" watchObservedRunningTime="2026-09-08 02:28:11.007923491 +0000 UTC m=+386.539568357" Sep 08 02:28:11.028083 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.027988 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-66b58bc974-fjz4g" podStartSLOduration=1.532071594 podStartE2EDuration="2.027959624s" totalImagesPullingTime="495.88803ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:28:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:28:11.024875311 +0000 UTC m=+386.556520169" watchObservedRunningTime="2026-09-08 02:28:11.027959624 +0000 UTC m=+386.559604488" Sep 08 02:28:11.059075 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.059037 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-q4nhg"] Sep 08 02:28:11.059246 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.059211 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-q4nhg" podUID="3c00376e-381e-43cb-8b44-c6c64323937b" containerName="authorino" containerID="cri-o://792b57a58b9725b78eca50911a165b1404877865378e4e3d3228223adcdb24ad" gracePeriod=30 Sep 08 02:28:11.246174 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.246144 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5cb866b664-rx898"] Sep 08 02:28:11.249745 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.249728 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cb866b664-rx898" Sep 08 02:28:11.253417 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.253387 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Sep 08 02:28:11.269891 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.269867 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cb866b664-rx898"] Sep 08 02:28:11.293349 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.293313 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75w54\" (UniqueName: \"kubernetes.io/projected/9bad9876-2d75-4d6f-9e94-4adcc2e0a441-kube-api-access-75w54\") pod \"authorino-5cb866b664-rx898\" (UID: \"9bad9876-2d75-4d6f-9e94-4adcc2e0a441\") " pod="kuadrant-system/authorino-5cb866b664-rx898" Sep 08 02:28:11.293559 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.293381 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9bad9876-2d75-4d6f-9e94-4adcc2e0a441-tls-cert\") pod \"authorino-5cb866b664-rx898\" (UID: \"9bad9876-2d75-4d6f-9e94-4adcc2e0a441\") " pod="kuadrant-system/authorino-5cb866b664-rx898" Sep 08 02:28:11.325130 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.325097 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-rf9sq" Sep 08 02:28:11.338628 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.338605 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-q4nhg" Sep 08 02:28:11.394390 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.394355 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frbq5\" (UniqueName: \"kubernetes.io/projected/dcbe2c77-26d2-46af-905a-5aecc5face16-kube-api-access-frbq5\") pod \"dcbe2c77-26d2-46af-905a-5aecc5face16\" (UID: \"dcbe2c77-26d2-46af-905a-5aecc5face16\") " Sep 08 02:28:11.394578 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.394413 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vjqd\" (UniqueName: \"kubernetes.io/projected/3c00376e-381e-43cb-8b44-c6c64323937b-kube-api-access-2vjqd\") pod \"3c00376e-381e-43cb-8b44-c6c64323937b\" (UID: \"3c00376e-381e-43cb-8b44-c6c64323937b\") " Sep 08 02:28:11.394647 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.394617 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75w54\" (UniqueName: \"kubernetes.io/projected/9bad9876-2d75-4d6f-9e94-4adcc2e0a441-kube-api-access-75w54\") pod \"authorino-5cb866b664-rx898\" (UID: \"9bad9876-2d75-4d6f-9e94-4adcc2e0a441\") " pod="kuadrant-system/authorino-5cb866b664-rx898" Sep 08 02:28:11.394700 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.394669 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9bad9876-2d75-4d6f-9e94-4adcc2e0a441-tls-cert\") pod \"authorino-5cb866b664-rx898\" (UID: \"9bad9876-2d75-4d6f-9e94-4adcc2e0a441\") " pod="kuadrant-system/authorino-5cb866b664-rx898" Sep 08 02:28:11.396586 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.396552 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcbe2c77-26d2-46af-905a-5aecc5face16-kube-api-access-frbq5" (OuterVolumeSpecName: "kube-api-access-frbq5") pod "dcbe2c77-26d2-46af-905a-5aecc5face16" (UID: "dcbe2c77-26d2-46af-905a-5aecc5face16"). InnerVolumeSpecName "kube-api-access-frbq5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 02:28:11.396696 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.396627 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c00376e-381e-43cb-8b44-c6c64323937b-kube-api-access-2vjqd" (OuterVolumeSpecName: "kube-api-access-2vjqd") pod "3c00376e-381e-43cb-8b44-c6c64323937b" (UID: "3c00376e-381e-43cb-8b44-c6c64323937b"). InnerVolumeSpecName "kube-api-access-2vjqd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 02:28:11.397331 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.397311 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9bad9876-2d75-4d6f-9e94-4adcc2e0a441-tls-cert\") pod \"authorino-5cb866b664-rx898\" (UID: \"9bad9876-2d75-4d6f-9e94-4adcc2e0a441\") " pod="kuadrant-system/authorino-5cb866b664-rx898" Sep 08 02:28:11.405654 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.405629 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75w54\" (UniqueName: \"kubernetes.io/projected/9bad9876-2d75-4d6f-9e94-4adcc2e0a441-kube-api-access-75w54\") pod \"authorino-5cb866b664-rx898\" (UID: \"9bad9876-2d75-4d6f-9e94-4adcc2e0a441\") " pod="kuadrant-system/authorino-5cb866b664-rx898" Sep 08 02:28:11.495488 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.495376 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frbq5\" (UniqueName: \"kubernetes.io/projected/dcbe2c77-26d2-46af-905a-5aecc5face16-kube-api-access-frbq5\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Sep 08 02:28:11.495488 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.495418 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vjqd\" (UniqueName: \"kubernetes.io/projected/3c00376e-381e-43cb-8b44-c6c64323937b-kube-api-access-2vjqd\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Sep 08 02:28:11.561942 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.561898 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cb866b664-rx898" Sep 08 02:28:11.838655 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.838629 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cb866b664-rx898"] Sep 08 02:28:11.840395 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:28:11.840356 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9bad9876_2d75_4d6f_9e94_4adcc2e0a441.slice/crio-588d394a52170f993cca3c0763b9b7e431503d0357a73fe4074f7523f2f56f15 WatchSource:0}: Error finding container 588d394a52170f993cca3c0763b9b7e431503d0357a73fe4074f7523f2f56f15: Status 404 returned error can't find the container with id 588d394a52170f993cca3c0763b9b7e431503d0357a73fe4074f7523f2f56f15 Sep 08 02:28:11.996313 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.996277 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cb866b664-rx898" event={"ID":"9bad9876-2d75-4d6f-9e94-4adcc2e0a441","Type":"ContainerStarted","Data":"588d394a52170f993cca3c0763b9b7e431503d0357a73fe4074f7523f2f56f15"} Sep 08 02:28:11.997579 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.997550 2560 generic.go:356] "Generic (PLEG): container finished" podID="dcbe2c77-26d2-46af-905a-5aecc5face16" containerID="794b29dae200842f0b00667903f024198350c754bfb2e48c70be6bc84497500d" exitCode=0 Sep 08 02:28:11.997694 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.997591 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-rf9sq" Sep 08 02:28:11.997694 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.997628 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-rf9sq" event={"ID":"dcbe2c77-26d2-46af-905a-5aecc5face16","Type":"ContainerDied","Data":"794b29dae200842f0b00667903f024198350c754bfb2e48c70be6bc84497500d"} Sep 08 02:28:11.997694 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.997659 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-rf9sq" event={"ID":"dcbe2c77-26d2-46af-905a-5aecc5face16","Type":"ContainerDied","Data":"304caddf56dd017d57001d8baa7cbccd360cf0da7997727de23d37a709e3810a"} Sep 08 02:28:11.997694 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.997679 2560 scope.go:117] "RemoveContainer" containerID="794b29dae200842f0b00667903f024198350c754bfb2e48c70be6bc84497500d" Sep 08 02:28:11.998958 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.998938 2560 generic.go:356] "Generic (PLEG): container finished" podID="3c00376e-381e-43cb-8b44-c6c64323937b" containerID="792b57a58b9725b78eca50911a165b1404877865378e4e3d3228223adcdb24ad" exitCode=0 Sep 08 02:28:11.999025 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.998972 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-q4nhg" Sep 08 02:28:11.999025 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.999006 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-q4nhg" event={"ID":"3c00376e-381e-43cb-8b44-c6c64323937b","Type":"ContainerDied","Data":"792b57a58b9725b78eca50911a165b1404877865378e4e3d3228223adcdb24ad"} Sep 08 02:28:11.999116 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:11.999058 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-q4nhg" event={"ID":"3c00376e-381e-43cb-8b44-c6c64323937b","Type":"ContainerDied","Data":"5dab9c4a59d37f51a8c44c96d45707004d53d1b21ee6ccce58e8953acc6ca830"} Sep 08 02:28:12.008012 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:12.007996 2560 scope.go:117] "RemoveContainer" containerID="794b29dae200842f0b00667903f024198350c754bfb2e48c70be6bc84497500d" Sep 08 02:28:12.008274 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:28:12.008252 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"794b29dae200842f0b00667903f024198350c754bfb2e48c70be6bc84497500d\": container with ID starting with 794b29dae200842f0b00667903f024198350c754bfb2e48c70be6bc84497500d not found: ID does not exist" containerID="794b29dae200842f0b00667903f024198350c754bfb2e48c70be6bc84497500d" Sep 08 02:28:12.008381 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:12.008280 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"794b29dae200842f0b00667903f024198350c754bfb2e48c70be6bc84497500d"} err="failed to get container status \"794b29dae200842f0b00667903f024198350c754bfb2e48c70be6bc84497500d\": rpc error: code = NotFound desc = could not find container \"794b29dae200842f0b00667903f024198350c754bfb2e48c70be6bc84497500d\": container with ID starting with 794b29dae200842f0b00667903f024198350c754bfb2e48c70be6bc84497500d not found: ID does not exist" Sep 08 02:28:12.008381 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:12.008296 2560 scope.go:117] "RemoveContainer" containerID="792b57a58b9725b78eca50911a165b1404877865378e4e3d3228223adcdb24ad" Sep 08 02:28:12.016011 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:12.015993 2560 scope.go:117] "RemoveContainer" containerID="792b57a58b9725b78eca50911a165b1404877865378e4e3d3228223adcdb24ad" Sep 08 02:28:12.016247 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:28:12.016230 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"792b57a58b9725b78eca50911a165b1404877865378e4e3d3228223adcdb24ad\": container with ID starting with 792b57a58b9725b78eca50911a165b1404877865378e4e3d3228223adcdb24ad not found: ID does not exist" containerID="792b57a58b9725b78eca50911a165b1404877865378e4e3d3228223adcdb24ad" Sep 08 02:28:12.016305 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:12.016251 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"792b57a58b9725b78eca50911a165b1404877865378e4e3d3228223adcdb24ad"} err="failed to get container status \"792b57a58b9725b78eca50911a165b1404877865378e4e3d3228223adcdb24ad\": rpc error: code = NotFound desc = could not find container \"792b57a58b9725b78eca50911a165b1404877865378e4e3d3228223adcdb24ad\": container with ID starting with 792b57a58b9725b78eca50911a165b1404877865378e4e3d3228223adcdb24ad not found: ID does not exist" Sep 08 02:28:12.024263 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:12.024242 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-rf9sq"] Sep 08 02:28:12.033135 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:12.033116 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-rf9sq"] Sep 08 02:28:12.043308 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:12.043286 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-q4nhg"] Sep 08 02:28:12.045909 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:12.045890 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-q4nhg"] Sep 08 02:28:13.008810 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:13.008772 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cb866b664-rx898" event={"ID":"9bad9876-2d75-4d6f-9e94-4adcc2e0a441","Type":"ContainerStarted","Data":"c5010960afbd3446ba2cec61ca065eb599ff1d2862c5f5dac45012143fc5d53e"} Sep 08 02:28:13.017730 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:13.017703 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c00376e-381e-43cb-8b44-c6c64323937b" path="/var/lib/kubelet/pods/3c00376e-381e-43cb-8b44-c6c64323937b/volumes" Sep 08 02:28:13.018029 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:13.018016 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcbe2c77-26d2-46af-905a-5aecc5face16" path="/var/lib/kubelet/pods/dcbe2c77-26d2-46af-905a-5aecc5face16/volumes" Sep 08 02:28:13.027113 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:13.027068 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5cb866b664-rx898" podStartSLOduration=1.024400666 podStartE2EDuration="2.02705356s" totalImagesPullingTime="1.002652894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:28:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:28:13.025403784 +0000 UTC m=+388.557048655" watchObservedRunningTime="2026-09-08 02:28:13.02705356 +0000 UTC m=+388.558698448" Sep 08 02:28:13.053593 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:13.053563 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66b58bc974-fjz4g"] Sep 08 02:28:13.054072 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:13.053710 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-66b58bc974-fjz4g" podUID="2783a172-49ac-4664-8ef8-083b08e7416a" containerName="authorino" containerID="cri-o://98fdccb064eb0eeed9acf0c453ec4ac1c4804b16e4dd8e2babde7a3e676f45a0" gracePeriod=30 Sep 08 02:28:13.307710 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:13.307682 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66b58bc974-fjz4g" Sep 08 02:28:13.311369 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:13.311344 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8t4cz\" (UniqueName: \"kubernetes.io/projected/2783a172-49ac-4664-8ef8-083b08e7416a-kube-api-access-8t4cz\") pod \"2783a172-49ac-4664-8ef8-083b08e7416a\" (UID: \"2783a172-49ac-4664-8ef8-083b08e7416a\") " Sep 08 02:28:13.313429 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:13.313403 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2783a172-49ac-4664-8ef8-083b08e7416a-kube-api-access-8t4cz" (OuterVolumeSpecName: "kube-api-access-8t4cz") pod "2783a172-49ac-4664-8ef8-083b08e7416a" (UID: "2783a172-49ac-4664-8ef8-083b08e7416a"). InnerVolumeSpecName "kube-api-access-8t4cz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 02:28:13.411987 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:13.411936 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8t4cz\" (UniqueName: \"kubernetes.io/projected/2783a172-49ac-4664-8ef8-083b08e7416a-kube-api-access-8t4cz\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Sep 08 02:28:13.947963 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:13.947936 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-bb7b75cc4-g7dxh" Sep 08 02:28:13.948289 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:13.948265 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-8jtjc" Sep 08 02:28:13.950345 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:13.950327 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-7d6487fcc8-gqr6x" Sep 08 02:28:14.017337 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:14.017301 2560 generic.go:356] "Generic (PLEG): container finished" podID="2783a172-49ac-4664-8ef8-083b08e7416a" containerID="98fdccb064eb0eeed9acf0c453ec4ac1c4804b16e4dd8e2babde7a3e676f45a0" exitCode=0 Sep 08 02:28:14.017544 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:14.017351 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66b58bc974-fjz4g" Sep 08 02:28:14.017544 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:14.017399 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66b58bc974-fjz4g" event={"ID":"2783a172-49ac-4664-8ef8-083b08e7416a","Type":"ContainerDied","Data":"98fdccb064eb0eeed9acf0c453ec4ac1c4804b16e4dd8e2babde7a3e676f45a0"} Sep 08 02:28:14.017544 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:14.017453 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66b58bc974-fjz4g" event={"ID":"2783a172-49ac-4664-8ef8-083b08e7416a","Type":"ContainerDied","Data":"310650c46dc31c0c35e369f45cd1df678f9a5bde3a2cc1713ad8f644f91b7652"} Sep 08 02:28:14.017544 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:14.017506 2560 scope.go:117] "RemoveContainer" containerID="98fdccb064eb0eeed9acf0c453ec4ac1c4804b16e4dd8e2babde7a3e676f45a0" Sep 08 02:28:14.027522 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:14.027496 2560 scope.go:117] "RemoveContainer" containerID="98fdccb064eb0eeed9acf0c453ec4ac1c4804b16e4dd8e2babde7a3e676f45a0" Sep 08 02:28:14.027822 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:28:14.027803 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98fdccb064eb0eeed9acf0c453ec4ac1c4804b16e4dd8e2babde7a3e676f45a0\": container with ID starting with 98fdccb064eb0eeed9acf0c453ec4ac1c4804b16e4dd8e2babde7a3e676f45a0 not found: ID does not exist" containerID="98fdccb064eb0eeed9acf0c453ec4ac1c4804b16e4dd8e2babde7a3e676f45a0" Sep 08 02:28:14.027876 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:14.027829 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98fdccb064eb0eeed9acf0c453ec4ac1c4804b16e4dd8e2babde7a3e676f45a0"} err="failed to get container status \"98fdccb064eb0eeed9acf0c453ec4ac1c4804b16e4dd8e2babde7a3e676f45a0\": rpc error: code = NotFound desc = could not find container \"98fdccb064eb0eeed9acf0c453ec4ac1c4804b16e4dd8e2babde7a3e676f45a0\": container with ID starting with 98fdccb064eb0eeed9acf0c453ec4ac1c4804b16e4dd8e2babde7a3e676f45a0 not found: ID does not exist" Sep 08 02:28:14.041797 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:14.041768 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66b58bc974-fjz4g"] Sep 08 02:28:14.044822 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:14.044800 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-66b58bc974-fjz4g"] Sep 08 02:28:15.017304 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:15.017272 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2783a172-49ac-4664-8ef8-083b08e7416a" path="/var/lib/kubelet/pods/2783a172-49ac-4664-8ef8-083b08e7416a/volumes" Sep 08 02:28:18.679766 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:18.679729 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-768897d8b6-82v2h"] Sep 08 02:28:18.680327 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:18.679983 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" podUID="6f25feac-0778-47fd-beac-dae58cbbb2ea" containerName="maas-api" containerID="cri-o://aee66e4604267ab0f9d67266fb5f0cec45e7b304c2a269213c040ad564fa679d" gracePeriod=30 Sep 08 02:28:18.928684 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:18.928659 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" Sep 08 02:28:18.962688 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:18.962613 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrmp7\" (UniqueName: \"kubernetes.io/projected/6f25feac-0778-47fd-beac-dae58cbbb2ea-kube-api-access-wrmp7\") pod \"6f25feac-0778-47fd-beac-dae58cbbb2ea\" (UID: \"6f25feac-0778-47fd-beac-dae58cbbb2ea\") " Sep 08 02:28:18.962688 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:18.962684 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f25feac-0778-47fd-beac-dae58cbbb2ea-maas-api-tls\") pod \"6f25feac-0778-47fd-beac-dae58cbbb2ea\" (UID: \"6f25feac-0778-47fd-beac-dae58cbbb2ea\") " Sep 08 02:28:18.962919 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:18.962711 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f25feac-0778-47fd-beac-dae58cbbb2ea-metrics-certs\") pod \"6f25feac-0778-47fd-beac-dae58cbbb2ea\" (UID: \"6f25feac-0778-47fd-beac-dae58cbbb2ea\") " Sep 08 02:28:18.962919 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:18.962787 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f25feac-0778-47fd-beac-dae58cbbb2ea-openshift-service-ca\") pod \"6f25feac-0778-47fd-beac-dae58cbbb2ea\" (UID: \"6f25feac-0778-47fd-beac-dae58cbbb2ea\") " Sep 08 02:28:18.963217 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:18.963192 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f25feac-0778-47fd-beac-dae58cbbb2ea-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6f25feac-0778-47fd-beac-dae58cbbb2ea" (UID: "6f25feac-0778-47fd-beac-dae58cbbb2ea"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 02:28:18.964898 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:18.964874 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f25feac-0778-47fd-beac-dae58cbbb2ea-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6f25feac-0778-47fd-beac-dae58cbbb2ea" (UID: "6f25feac-0778-47fd-beac-dae58cbbb2ea"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 02:28:18.965006 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:18.964945 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f25feac-0778-47fd-beac-dae58cbbb2ea-kube-api-access-wrmp7" (OuterVolumeSpecName: "kube-api-access-wrmp7") pod "6f25feac-0778-47fd-beac-dae58cbbb2ea" (UID: "6f25feac-0778-47fd-beac-dae58cbbb2ea"). InnerVolumeSpecName "kube-api-access-wrmp7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 02:28:18.965120 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:18.965104 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f25feac-0778-47fd-beac-dae58cbbb2ea-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6f25feac-0778-47fd-beac-dae58cbbb2ea" (UID: "6f25feac-0778-47fd-beac-dae58cbbb2ea"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 02:28:19.043769 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:19.043737 2560 generic.go:356] "Generic (PLEG): container finished" podID="6f25feac-0778-47fd-beac-dae58cbbb2ea" containerID="aee66e4604267ab0f9d67266fb5f0cec45e7b304c2a269213c040ad564fa679d" exitCode=0 Sep 08 02:28:19.043920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:19.043810 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" Sep 08 02:28:19.043920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:19.043819 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" event={"ID":"6f25feac-0778-47fd-beac-dae58cbbb2ea","Type":"ContainerDied","Data":"aee66e4604267ab0f9d67266fb5f0cec45e7b304c2a269213c040ad564fa679d"} Sep 08 02:28:19.043920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:19.043881 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-768897d8b6-82v2h" event={"ID":"6f25feac-0778-47fd-beac-dae58cbbb2ea","Type":"ContainerDied","Data":"64f424b1fac0aa145bd4be70fd67e42f26a886fda16ade39406c84684f4a9e61"} Sep 08 02:28:19.043920 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:19.043910 2560 scope.go:117] "RemoveContainer" containerID="aee66e4604267ab0f9d67266fb5f0cec45e7b304c2a269213c040ad564fa679d" Sep 08 02:28:19.053881 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:19.053856 2560 scope.go:117] "RemoveContainer" containerID="aee66e4604267ab0f9d67266fb5f0cec45e7b304c2a269213c040ad564fa679d" Sep 08 02:28:19.054220 ip-10-0-141-33 kubenswrapper[2560]: E0908 02:28:19.054198 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aee66e4604267ab0f9d67266fb5f0cec45e7b304c2a269213c040ad564fa679d\": container with ID starting with aee66e4604267ab0f9d67266fb5f0cec45e7b304c2a269213c040ad564fa679d not found: ID does not exist" containerID="aee66e4604267ab0f9d67266fb5f0cec45e7b304c2a269213c040ad564fa679d" Sep 08 02:28:19.054322 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:19.054226 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aee66e4604267ab0f9d67266fb5f0cec45e7b304c2a269213c040ad564fa679d"} err="failed to get container status \"aee66e4604267ab0f9d67266fb5f0cec45e7b304c2a269213c040ad564fa679d\": rpc error: code = NotFound desc = could not find container \"aee66e4604267ab0f9d67266fb5f0cec45e7b304c2a269213c040ad564fa679d\": container with ID starting with aee66e4604267ab0f9d67266fb5f0cec45e7b304c2a269213c040ad564fa679d not found: ID does not exist" Sep 08 02:28:19.064296 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:19.064273 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrmp7\" (UniqueName: \"kubernetes.io/projected/6f25feac-0778-47fd-beac-dae58cbbb2ea-kube-api-access-wrmp7\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Sep 08 02:28:19.064296 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:19.064295 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f25feac-0778-47fd-beac-dae58cbbb2ea-maas-api-tls\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Sep 08 02:28:19.064296 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:19.064305 2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f25feac-0778-47fd-beac-dae58cbbb2ea-metrics-certs\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Sep 08 02:28:19.065193 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:19.064317 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f25feac-0778-47fd-beac-dae58cbbb2ea-openshift-service-ca\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Sep 08 02:28:19.067427 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:19.067402 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-768897d8b6-82v2h"] Sep 08 02:28:19.069286 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:19.069265 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-768897d8b6-82v2h"] Sep 08 02:28:21.017881 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:21.017849 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f25feac-0778-47fd-beac-dae58cbbb2ea" path="/var/lib/kubelet/pods/6f25feac-0778-47fd-beac-dae58cbbb2ea/volumes" Sep 08 02:28:33.951542 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:28:33.951510 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-85d49875c6-xnn5f" Sep 08 02:29:03.421454 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.421414 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg"] Sep 08 02:29:03.421905 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.421839 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f25feac-0778-47fd-beac-dae58cbbb2ea" containerName="maas-api" Sep 08 02:29:03.421905 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.421852 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f25feac-0778-47fd-beac-dae58cbbb2ea" containerName="maas-api" Sep 08 02:29:03.421905 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.421862 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcbe2c77-26d2-46af-905a-5aecc5face16" containerName="authorino" Sep 08 02:29:03.421905 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.421867 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcbe2c77-26d2-46af-905a-5aecc5face16" containerName="authorino" Sep 08 02:29:03.421905 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.421874 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2783a172-49ac-4664-8ef8-083b08e7416a" containerName="authorino" Sep 08 02:29:03.421905 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.421880 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2783a172-49ac-4664-8ef8-083b08e7416a" containerName="authorino" Sep 08 02:29:03.421905 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.421889 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c00376e-381e-43cb-8b44-c6c64323937b" containerName="authorino" Sep 08 02:29:03.421905 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.421894 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c00376e-381e-43cb-8b44-c6c64323937b" containerName="authorino" Sep 08 02:29:03.422142 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.421952 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c00376e-381e-43cb-8b44-c6c64323937b" containerName="authorino" Sep 08 02:29:03.422142 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.421962 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcbe2c77-26d2-46af-905a-5aecc5face16" containerName="authorino" Sep 08 02:29:03.422142 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.421970 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2783a172-49ac-4664-8ef8-083b08e7416a" containerName="authorino" Sep 08 02:29:03.422142 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.421976 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f25feac-0778-47fd-beac-dae58cbbb2ea" containerName="maas-api" Sep 08 02:29:03.426812 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.426789 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:03.431590 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.431570 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 08 02:29:03.431746 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.431722 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 08 02:29:03.431878 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.431657 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-q24xq\"" Sep 08 02:29:03.431878 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.431643 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Sep 08 02:29:03.436487 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.436452 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg"] Sep 08 02:29:03.459078 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.459054 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaed51dd-36c5-4414-ab2b-878a38893d98-model-cache\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg\" (UID: \"aaed51dd-36c5-4414-ab2b-878a38893d98\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:03.459078 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.459085 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaed51dd-36c5-4414-ab2b-878a38893d98-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg\" (UID: \"aaed51dd-36c5-4414-ab2b-878a38893d98\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:03.459270 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.459110 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaed51dd-36c5-4414-ab2b-878a38893d98-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg\" (UID: \"aaed51dd-36c5-4414-ab2b-878a38893d98\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:03.459270 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.459221 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqq4n\" (UniqueName: \"kubernetes.io/projected/aaed51dd-36c5-4414-ab2b-878a38893d98-kube-api-access-sqq4n\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg\" (UID: \"aaed51dd-36c5-4414-ab2b-878a38893d98\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:03.459377 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.459266 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaed51dd-36c5-4414-ab2b-878a38893d98-home\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg\" (UID: \"aaed51dd-36c5-4414-ab2b-878a38893d98\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:03.459430 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.459368 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaed51dd-36c5-4414-ab2b-878a38893d98-dshm\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg\" (UID: \"aaed51dd-36c5-4414-ab2b-878a38893d98\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:03.560859 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.560825 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqq4n\" (UniqueName: \"kubernetes.io/projected/aaed51dd-36c5-4414-ab2b-878a38893d98-kube-api-access-sqq4n\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg\" (UID: \"aaed51dd-36c5-4414-ab2b-878a38893d98\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:03.561042 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.560873 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaed51dd-36c5-4414-ab2b-878a38893d98-home\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg\" (UID: \"aaed51dd-36c5-4414-ab2b-878a38893d98\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:03.561042 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.560897 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaed51dd-36c5-4414-ab2b-878a38893d98-dshm\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg\" (UID: \"aaed51dd-36c5-4414-ab2b-878a38893d98\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:03.561042 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.560945 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaed51dd-36c5-4414-ab2b-878a38893d98-model-cache\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg\" (UID: \"aaed51dd-36c5-4414-ab2b-878a38893d98\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:03.561042 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.560965 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaed51dd-36c5-4414-ab2b-878a38893d98-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg\" (UID: \"aaed51dd-36c5-4414-ab2b-878a38893d98\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:03.561042 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.561000 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaed51dd-36c5-4414-ab2b-878a38893d98-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg\" (UID: \"aaed51dd-36c5-4414-ab2b-878a38893d98\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:03.561322 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.561296 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaed51dd-36c5-4414-ab2b-878a38893d98-home\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg\" (UID: \"aaed51dd-36c5-4414-ab2b-878a38893d98\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:03.561400 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.561358 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaed51dd-36c5-4414-ab2b-878a38893d98-model-cache\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg\" (UID: \"aaed51dd-36c5-4414-ab2b-878a38893d98\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:03.561400 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.561378 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaed51dd-36c5-4414-ab2b-878a38893d98-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg\" (UID: \"aaed51dd-36c5-4414-ab2b-878a38893d98\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:03.563241 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.563218 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaed51dd-36c5-4414-ab2b-878a38893d98-dshm\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg\" (UID: \"aaed51dd-36c5-4414-ab2b-878a38893d98\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:03.563531 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.563512 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaed51dd-36c5-4414-ab2b-878a38893d98-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg\" (UID: \"aaed51dd-36c5-4414-ab2b-878a38893d98\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:03.569656 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.569636 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqq4n\" (UniqueName: \"kubernetes.io/projected/aaed51dd-36c5-4414-ab2b-878a38893d98-kube-api-access-sqq4n\") pod \"e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg\" (UID: \"aaed51dd-36c5-4414-ab2b-878a38893d98\") " pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:03.739903 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:03.739818 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:04.020823 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:04.020796 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg"] Sep 08 02:29:04.021874 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:29:04.021846 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaed51dd_36c5_4414_ab2b_878a38893d98.slice/crio-d99dc512af9321f36706418e66fe3545acd9f84864dfdd6df5579d1ccf71a21e WatchSource:0}: Error finding container d99dc512af9321f36706418e66fe3545acd9f84864dfdd6df5579d1ccf71a21e: Status 404 returned error can't find the container with id d99dc512af9321f36706418e66fe3545acd9f84864dfdd6df5579d1ccf71a21e Sep 08 02:29:04.031972 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:04.031950 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv"] Sep 08 02:29:04.293117 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:04.293033 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" event={"ID":"aaed51dd-36c5-4414-ab2b-878a38893d98","Type":"ContainerStarted","Data":"d99dc512af9321f36706418e66fe3545acd9f84864dfdd6df5579d1ccf71a21e"} Sep 08 02:29:06.305346 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:06.305305 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" event={"ID":"aaed51dd-36c5-4414-ab2b-878a38893d98","Type":"ContainerStarted","Data":"6eec5682adcdc4cad44036375a0c780e864a53410022e5ea7ffbfa5d47950e62"} Sep 08 02:29:06.328324 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:06.328276 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" podStartSLOduration=2.077706053 podStartE2EDuration="3.328259527s" totalImagesPullingTime="1.250553474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:29:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:29:06.32702635 +0000 UTC m=+441.858671219" watchObservedRunningTime="2026-09-08 02:29:06.328259527 +0000 UTC m=+441.859904392" Sep 08 02:29:06.740950 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:06.740910 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:13.033975 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.033939 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t"] Sep 08 02:29:13.038217 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.038193 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:13.041063 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.041040 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Sep 08 02:29:13.048039 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.048006 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t"] Sep 08 02:29:13.151114 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.151082 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30f8dc58-ae9c-4621-b38f-306b104cce33-tmp-dir\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-7lb7t\" (UID: \"30f8dc58-ae9c-4621-b38f-306b104cce33\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:13.151291 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.151137 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30f8dc58-ae9c-4621-b38f-306b104cce33-dshm\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-7lb7t\" (UID: \"30f8dc58-ae9c-4621-b38f-306b104cce33\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:13.151291 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.151164 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30f8dc58-ae9c-4621-b38f-306b104cce33-model-cache\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-7lb7t\" (UID: \"30f8dc58-ae9c-4621-b38f-306b104cce33\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:13.151291 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.151221 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdwsn\" (UniqueName: \"kubernetes.io/projected/30f8dc58-ae9c-4621-b38f-306b104cce33-kube-api-access-gdwsn\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-7lb7t\" (UID: \"30f8dc58-ae9c-4621-b38f-306b104cce33\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:13.151291 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.151247 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30f8dc58-ae9c-4621-b38f-306b104cce33-home\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-7lb7t\" (UID: \"30f8dc58-ae9c-4621-b38f-306b104cce33\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:13.151291 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.151273 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30f8dc58-ae9c-4621-b38f-306b104cce33-tls-certs\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-7lb7t\" (UID: \"30f8dc58-ae9c-4621-b38f-306b104cce33\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:13.252226 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.252185 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdwsn\" (UniqueName: \"kubernetes.io/projected/30f8dc58-ae9c-4621-b38f-306b104cce33-kube-api-access-gdwsn\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-7lb7t\" (UID: \"30f8dc58-ae9c-4621-b38f-306b104cce33\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:13.252226 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.252227 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30f8dc58-ae9c-4621-b38f-306b104cce33-home\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-7lb7t\" (UID: \"30f8dc58-ae9c-4621-b38f-306b104cce33\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:13.252431 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.252267 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30f8dc58-ae9c-4621-b38f-306b104cce33-tls-certs\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-7lb7t\" (UID: \"30f8dc58-ae9c-4621-b38f-306b104cce33\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:13.252431 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.252310 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30f8dc58-ae9c-4621-b38f-306b104cce33-tmp-dir\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-7lb7t\" (UID: \"30f8dc58-ae9c-4621-b38f-306b104cce33\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:13.252431 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.252361 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30f8dc58-ae9c-4621-b38f-306b104cce33-dshm\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-7lb7t\" (UID: \"30f8dc58-ae9c-4621-b38f-306b104cce33\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:13.252431 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.252385 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30f8dc58-ae9c-4621-b38f-306b104cce33-model-cache\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-7lb7t\" (UID: \"30f8dc58-ae9c-4621-b38f-306b104cce33\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:13.252698 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.252680 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30f8dc58-ae9c-4621-b38f-306b104cce33-home\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-7lb7t\" (UID: \"30f8dc58-ae9c-4621-b38f-306b104cce33\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:13.252762 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.252724 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30f8dc58-ae9c-4621-b38f-306b104cce33-tmp-dir\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-7lb7t\" (UID: \"30f8dc58-ae9c-4621-b38f-306b104cce33\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:13.252866 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.252844 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30f8dc58-ae9c-4621-b38f-306b104cce33-model-cache\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-7lb7t\" (UID: \"30f8dc58-ae9c-4621-b38f-306b104cce33\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:13.254723 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.254698 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30f8dc58-ae9c-4621-b38f-306b104cce33-dshm\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-7lb7t\" (UID: \"30f8dc58-ae9c-4621-b38f-306b104cce33\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:13.254849 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.254831 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30f8dc58-ae9c-4621-b38f-306b104cce33-tls-certs\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-7lb7t\" (UID: \"30f8dc58-ae9c-4621-b38f-306b104cce33\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:13.262299 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.262261 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdwsn\" (UniqueName: \"kubernetes.io/projected/30f8dc58-ae9c-4621-b38f-306b104cce33-kube-api-access-gdwsn\") pod \"e2e-distinct-simulated-kserve-5d57fc946d-7lb7t\" (UID: \"30f8dc58-ae9c-4621-b38f-306b104cce33\") " pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:13.349669 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.349635 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:13.622566 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.622538 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t"] Sep 08 02:29:13.624763 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:29:13.624736 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30f8dc58_ae9c_4621_b38f_306b104cce33.slice/crio-ab73b4a5ad0e905c0cb1bc3038d2c7a2b4c42cc381073019fb5ab3550656f63a WatchSource:0}: Error finding container ab73b4a5ad0e905c0cb1bc3038d2c7a2b4c42cc381073019fb5ab3550656f63a: Status 404 returned error can't find the container with id ab73b4a5ad0e905c0cb1bc3038d2c7a2b4c42cc381073019fb5ab3550656f63a Sep 08 02:29:13.740451 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.740413 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:13.753255 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:13.753221 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:14.140953 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:14.140916 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv"] Sep 08 02:29:14.346938 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:14.346894 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" event={"ID":"30f8dc58-ae9c-4621-b38f-306b104cce33","Type":"ContainerStarted","Data":"95f8d653c6a6900ab9baa8239f209bd6c27b1dff4781e8f05efdaa7a22c8dbd5"} Sep 08 02:29:14.347101 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:14.346965 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" event={"ID":"30f8dc58-ae9c-4621-b38f-306b104cce33","Type":"ContainerStarted","Data":"ab73b4a5ad0e905c0cb1bc3038d2c7a2b4c42cc381073019fb5ab3550656f63a"} Sep 08 02:29:14.350114 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:14.350083 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:14.359429 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:14.359401 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg" Sep 08 02:29:14.375404 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:14.375349 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" podStartSLOduration=1.200964972 podStartE2EDuration="1.375332546s" totalImagesPullingTime="174.367574ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:29:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:29:14.372268419 +0000 UTC m=+449.903913296" watchObservedRunningTime="2026-09-08 02:29:14.375332546 +0000 UTC m=+449.906977404" Sep 08 02:29:23.350727 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:23.350693 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:23.362923 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:23.362895 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:23.403215 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:23.403191 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5d57fc946d-7lb7t" Sep 08 02:29:29.537873 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:29.537838 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv"] Sep 08 02:29:44.638486 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:44.638431 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv"] Sep 08 02:29:50.943038 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:29:50.943001 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv"] Sep 08 02:30:00.132802 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:30:00.132763 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv"] Sep 08 02:30:00.151186 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:30:00.151153 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29813910-lwnr9"] Sep 08 02:30:00.155216 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:30:00.155200 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813910-lwnr9" Sep 08 02:30:00.158567 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:30:00.158545 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lz6h\"" Sep 08 02:30:00.162499 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:30:00.162451 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29813910-lwnr9"] Sep 08 02:30:00.166402 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:30:00.166383 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813910-lwnr9" Sep 08 02:30:00.439694 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:30:00.439667 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29813910-lwnr9"] Sep 08 02:30:00.441481 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:30:00.441442 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c112e1e_cc6b_40f6_bb2c_e79557dbb217.slice/crio-9d0570d92532485368f0299c6f66158bc1e7117f2f49758c17a2b29f2d2b6e98 WatchSource:0}: Error finding container 9d0570d92532485368f0299c6f66158bc1e7117f2f49758c17a2b29f2d2b6e98: Status 404 returned error can't find the container with id 9d0570d92532485368f0299c6f66158bc1e7117f2f49758c17a2b29f2d2b6e98 Sep 08 02:30:00.594129 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:30:00.594096 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813910-lwnr9" event={"ID":"0c112e1e-cc6b-40f6-bb2c-e79557dbb217","Type":"ContainerStarted","Data":"9d0570d92532485368f0299c6f66158bc1e7117f2f49758c17a2b29f2d2b6e98"} Sep 08 02:30:03.615475 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:30:03.615427 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813910-lwnr9" event={"ID":"0c112e1e-cc6b-40f6-bb2c-e79557dbb217","Type":"ContainerStarted","Data":"62d2e0ed7259b0e025cf4151155feb62ac44ad847b14dd4c1727baadfa51b9be"} Sep 08 02:30:03.634828 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:30:03.634783 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813910-lwnr9" podStartSLOduration=1.248827229 podStartE2EDuration="3.634769202s" totalImagesPullingTime="2.385941973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:30:03.633259953 +0000 UTC m=+499.164904818" watchObservedRunningTime="2026-09-08 02:30:03.634769202 +0000 UTC m=+499.166414067" Sep 08 02:30:04.622694 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:30:04.622658 2560 generic.go:356] "Generic (PLEG): container finished" podID="0c112e1e-cc6b-40f6-bb2c-e79557dbb217" containerID="62d2e0ed7259b0e025cf4151155feb62ac44ad847b14dd4c1727baadfa51b9be" exitCode=0 Sep 08 02:30:04.623129 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:30:04.622727 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813910-lwnr9" event={"ID":"0c112e1e-cc6b-40f6-bb2c-e79557dbb217","Type":"ContainerDied","Data":"62d2e0ed7259b0e025cf4151155feb62ac44ad847b14dd4c1727baadfa51b9be"} Sep 08 02:30:05.760456 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:30:05.760432 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813910-lwnr9" Sep 08 02:30:06.633643 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:30:06.633616 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813910-lwnr9" Sep 08 02:30:06.633814 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:30:06.633616 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813910-lwnr9" event={"ID":"0c112e1e-cc6b-40f6-bb2c-e79557dbb217","Type":"ContainerDied","Data":"9d0570d92532485368f0299c6f66158bc1e7117f2f49758c17a2b29f2d2b6e98"} Sep 08 02:30:06.633814 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:30:06.633741 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d0570d92532485368f0299c6f66158bc1e7117f2f49758c17a2b29f2d2b6e98" Sep 08 02:30:16.639005 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:30:16.638968 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-7zwzv"] Sep 08 02:36:21.269626 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:21.269596 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-657f59599b-dpqc5_18bd4cc6-3225-4611-971f-611be27802ee/manager/0.log" Sep 08 02:36:21.636217 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:21.636184 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5cb866b664-rx898_9bad9876-2d75-4d6f-9e94-4adcc2e0a441/authorino/0.log" Sep 08 02:36:27.643110 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:27.643081 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-85d49875c6-xnn5f_c45c3758-5159-43b1-a8b0-58c14735ae03/manager/0.log" Sep 08 02:36:27.785727 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:27.785695 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-54765d59fd-rvkzh_dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435/manager/0.log" Sep 08 02:36:28.120169 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:28.120141 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-657f59599b-dpqc5_18bd4cc6-3225-4611-971f-611be27802ee/manager/0.log" Sep 08 02:36:28.239802 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:28.239774 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-7d6487fcc8-gqr6x_2c57646e-c736-4ca0-9f9a-8c271a1e8169/server/0.log" Sep 08 02:36:28.377511 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:28.377427 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-bb7b75cc4-g7dxh_6ba0375c-8d15-4d23-b19e-63c45bfa0e2a/manager/0.log" Sep 08 02:36:28.862297 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:28.862263 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29813910-lwnr9_0c112e1e-cc6b-40f6-bb2c-e79557dbb217/cleanup/0.log" Sep 08 02:36:29.843867 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:29.843835 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5cb866b664-rx898_9bad9876-2d75-4d6f-9e94-4adcc2e0a441/authorino/0.log" Sep 08 02:36:30.086199 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:30.086166 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-ffcddf5cd-5zb86_30ab6d94-b77f-4c9c-9d7d-a29fc93bc6b3/manager/0.log" Sep 08 02:36:30.448845 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:30.448815 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-7zwzv_f6c0f248-c2cd-4596-961c-606065a8deef/limitador/0.log" Sep 08 02:36:31.195301 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:31.195268 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65f559df56-cqzn5_7b7b6a75-411c-41dd-b5fe-e61ff6172318/kube-auth-proxy/0.log" Sep 08 02:36:31.794556 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:31.794522 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-8jtjc_eedd08c8-e8e8-4464-9da0-c522a2d09a2b/payload-processing/0.log" Sep 08 02:36:31.908342 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:31.908299 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d6c5474d8-9wrc6_55e533a6-3267-4b11-966b-677e70829961/router/0.log" Sep 08 02:36:32.259227 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:32.259196 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-5fc5ddd44f-6qjxg_aaed51dd-36c5-4414-ab2b-878a38893d98/main/0.log" Sep 08 02:36:32.372417 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:32.372390 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5d57fc946d-7lb7t_30f8dc58-ae9c-4621-b38f-306b104cce33/main/0.log" Sep 08 02:36:38.840393 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:38.840312 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jf4m6/must-gather-2zc45"] Sep 08 02:36:38.840788 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:38.840760 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c112e1e-cc6b-40f6-bb2c-e79557dbb217" containerName="cleanup" Sep 08 02:36:38.840788 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:38.840773 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c112e1e-cc6b-40f6-bb2c-e79557dbb217" containerName="cleanup" Sep 08 02:36:38.840895 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:38.840867 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c112e1e-cc6b-40f6-bb2c-e79557dbb217" containerName="cleanup" Sep 08 02:36:38.844066 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:38.844047 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jf4m6/must-gather-2zc45" Sep 08 02:36:38.847140 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:38.847115 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jf4m6\"/\"openshift-service-ca.crt\"" Sep 08 02:36:38.847140 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:38.847128 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jf4m6\"/\"kube-root-ca.crt\"" Sep 08 02:36:38.848639 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:38.848584 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jf4m6\"/\"default-dockercfg-qzs9c\"" Sep 08 02:36:38.862679 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:38.862655 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jf4m6/must-gather-2zc45"] Sep 08 02:36:38.999826 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:38.999787 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p27kt\" (UniqueName: \"kubernetes.io/projected/a2bca818-afed-41a2-a698-ab4b472d09f3-kube-api-access-p27kt\") pod \"must-gather-2zc45\" (UID: \"a2bca818-afed-41a2-a698-ab4b472d09f3\") " pod="openshift-must-gather-jf4m6/must-gather-2zc45" Sep 08 02:36:38.999985 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:38.999858 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a2bca818-afed-41a2-a698-ab4b472d09f3-must-gather-output\") pod \"must-gather-2zc45\" (UID: \"a2bca818-afed-41a2-a698-ab4b472d09f3\") " pod="openshift-must-gather-jf4m6/must-gather-2zc45" Sep 08 02:36:39.101239 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:39.101214 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a2bca818-afed-41a2-a698-ab4b472d09f3-must-gather-output\") pod \"must-gather-2zc45\" (UID: \"a2bca818-afed-41a2-a698-ab4b472d09f3\") " pod="openshift-must-gather-jf4m6/must-gather-2zc45" Sep 08 02:36:39.101383 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:39.101298 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p27kt\" (UniqueName: \"kubernetes.io/projected/a2bca818-afed-41a2-a698-ab4b472d09f3-kube-api-access-p27kt\") pod \"must-gather-2zc45\" (UID: \"a2bca818-afed-41a2-a698-ab4b472d09f3\") " pod="openshift-must-gather-jf4m6/must-gather-2zc45" Sep 08 02:36:39.101584 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:39.101564 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a2bca818-afed-41a2-a698-ab4b472d09f3-must-gather-output\") pod \"must-gather-2zc45\" (UID: \"a2bca818-afed-41a2-a698-ab4b472d09f3\") " pod="openshift-must-gather-jf4m6/must-gather-2zc45" Sep 08 02:36:39.115368 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:39.115340 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p27kt\" (UniqueName: \"kubernetes.io/projected/a2bca818-afed-41a2-a698-ab4b472d09f3-kube-api-access-p27kt\") pod \"must-gather-2zc45\" (UID: \"a2bca818-afed-41a2-a698-ab4b472d09f3\") " pod="openshift-must-gather-jf4m6/must-gather-2zc45" Sep 08 02:36:39.178147 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:39.178102 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jf4m6/must-gather-2zc45" Sep 08 02:36:39.449784 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:39.449761 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jf4m6/must-gather-2zc45"] Sep 08 02:36:39.451147 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:36:39.451121 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2bca818_afed_41a2_a698_ab4b472d09f3.slice/crio-dbcc24cf2faae9307ec541cfec05812668b532d3d391765a980b6fd36a13bf93 WatchSource:0}: Error finding container dbcc24cf2faae9307ec541cfec05812668b532d3d391765a980b6fd36a13bf93: Status 404 returned error can't find the container with id dbcc24cf2faae9307ec541cfec05812668b532d3d391765a980b6fd36a13bf93 Sep 08 02:36:39.452960 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:39.452940 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 02:36:39.607284 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:39.607249 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jf4m6/must-gather-2zc45" event={"ID":"a2bca818-afed-41a2-a698-ab4b472d09f3","Type":"ContainerStarted","Data":"dbcc24cf2faae9307ec541cfec05812668b532d3d391765a980b6fd36a13bf93"} Sep 08 02:36:40.618200 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:40.618161 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jf4m6/must-gather-2zc45" event={"ID":"a2bca818-afed-41a2-a698-ab4b472d09f3","Type":"ContainerStarted","Data":"5c3a52a556f9b75fa50be6f078a35cb92a25a31a45a6151a79230d331b3a86a3"} Sep 08 02:36:40.618696 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:40.618210 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jf4m6/must-gather-2zc45" event={"ID":"a2bca818-afed-41a2-a698-ab4b472d09f3","Type":"ContainerStarted","Data":"19e1a279f05d1bc9a0141e93b6bd64909e804125192f6e9f5640ba224a6f82ab"} Sep 08 02:36:40.638100 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:40.638045 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jf4m6/must-gather-2zc45" podStartSLOduration=1.842523018 podStartE2EDuration="2.638028966s" totalImagesPullingTime="795.505948ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 02:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:36:40.636511847 +0000 UTC m=+896.168156716" watchObservedRunningTime="2026-09-08 02:36:40.638028966 +0000 UTC m=+896.169673832" Sep 08 02:36:41.914796 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:41.914766 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pmw7v_0af6038b-8bad-4d6a-9478-083edfbc974a/global-pull-secret-syncer/0.log" Sep 08 02:36:41.991849 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:41.991822 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gp9px_550f906c-4554-42d1-9ef7-1922f2501656/konnectivity-agent/0.log" Sep 08 02:36:42.071288 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:42.071257 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-33.ec2.internal_a1d443509e8525aa8dff4d1a40896247/haproxy/0.log" Sep 08 02:36:46.610132 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:46.610093 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5cb866b664-rx898_9bad9876-2d75-4d6f-9e94-4adcc2e0a441/authorino/0.log" Sep 08 02:36:46.688041 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:46.687926 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-ffcddf5cd-5zb86_30ab6d94-b77f-4c9c-9d7d-a29fc93bc6b3/manager/0.log" Sep 08 02:36:46.800661 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:46.800634 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-7zwzv_f6c0f248-c2cd-4596-961c-606065a8deef/limitador/0.log" Sep 08 02:36:48.219554 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:48.219502 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cc563bc7-8836-4951-8ffa-3517967f4cab/alertmanager/0.log" Sep 08 02:36:48.257449 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:48.256986 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cc563bc7-8836-4951-8ffa-3517967f4cab/config-reloader/0.log" Sep 08 02:36:48.294104 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:48.294064 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cc563bc7-8836-4951-8ffa-3517967f4cab/kube-rbac-proxy-web/0.log" Sep 08 02:36:48.330146 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:48.330113 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cc563bc7-8836-4951-8ffa-3517967f4cab/kube-rbac-proxy/0.log" Sep 08 02:36:48.359689 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:48.359643 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cc563bc7-8836-4951-8ffa-3517967f4cab/kube-rbac-proxy-metric/0.log" Sep 08 02:36:48.393706 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:48.393619 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cc563bc7-8836-4951-8ffa-3517967f4cab/prom-label-proxy/0.log" Sep 08 02:36:48.422692 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:48.422650 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_cc563bc7-8836-4951-8ffa-3517967f4cab/init-config-reloader/0.log" Sep 08 02:36:48.587639 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:48.587606 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-697fdf7c8b-7stmv_bbe38cf3-5b49-4c35-b3cf-31e5b82172e8/metrics-server/0.log" Sep 08 02:36:48.826531 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:48.826040 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-627dq_d8c7193e-0ea8-47a5-9989-1a413144bb8a/node-exporter/0.log" Sep 08 02:36:48.864793 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:48.864761 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-627dq_d8c7193e-0ea8-47a5-9989-1a413144bb8a/kube-rbac-proxy/0.log" Sep 08 02:36:48.914545 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:48.914516 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-627dq_d8c7193e-0ea8-47a5-9989-1a413144bb8a/init-textfile/0.log" Sep 08 02:36:49.134541 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:49.134508 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6c2dd45b-924d-40a5-81c0-ed2f72d63ab0/prometheus/0.log" Sep 08 02:36:49.160602 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:49.160519 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6c2dd45b-924d-40a5-81c0-ed2f72d63ab0/config-reloader/0.log" Sep 08 02:36:49.193315 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:49.193271 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6c2dd45b-924d-40a5-81c0-ed2f72d63ab0/thanos-sidecar/0.log" Sep 08 02:36:49.216428 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:49.216345 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6c2dd45b-924d-40a5-81c0-ed2f72d63ab0/kube-rbac-proxy-web/0.log" Sep 08 02:36:49.243933 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:49.243875 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6c2dd45b-924d-40a5-81c0-ed2f72d63ab0/kube-rbac-proxy/0.log" Sep 08 02:36:49.265941 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:49.265901 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6c2dd45b-924d-40a5-81c0-ed2f72d63ab0/kube-rbac-proxy-thanos/0.log" Sep 08 02:36:49.290794 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:49.290757 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6c2dd45b-924d-40a5-81c0-ed2f72d63ab0/init-config-reloader/0.log" Sep 08 02:36:49.335998 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:49.335965 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-7qxms_3deb3cf9-9352-4589-82bd-3ea0f048181a/prometheus-operator/0.log" Sep 08 02:36:49.362654 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:49.362625 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-7qxms_3deb3cf9-9352-4589-82bd-3ea0f048181a/kube-rbac-proxy/0.log" Sep 08 02:36:49.389909 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:49.389834 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-7zh78_045f37f9-6764-4abf-9545-bea7e4137bf5/prometheus-operator-admission-webhook/0.log" Sep 08 02:36:49.430972 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:49.430911 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6589c8c8fc-gjf6n_38ac877f-56e9-418b-9fa0-0456fbc73c7e/telemeter-client/0.log" Sep 08 02:36:49.460291 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:49.460264 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6589c8c8fc-gjf6n_38ac877f-56e9-418b-9fa0-0456fbc73c7e/reload/0.log" Sep 08 02:36:49.486022 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:49.485965 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6589c8c8fc-gjf6n_38ac877f-56e9-418b-9fa0-0456fbc73c7e/kube-rbac-proxy/0.log" Sep 08 02:36:51.700942 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:51.700903 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb"] Sep 08 02:36:51.704972 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:51.704945 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb" Sep 08 02:36:51.716329 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:51.716295 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb"] Sep 08 02:36:51.747038 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:51.746986 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7bd1b273-db16-460b-89a8-94c4ebc0404e-proc\") pod \"perf-node-gather-daemonset-nspfb\" (UID: \"7bd1b273-db16-460b-89a8-94c4ebc0404e\") " pod="openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb" Sep 08 02:36:51.747279 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:51.747260 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7bd1b273-db16-460b-89a8-94c4ebc0404e-sys\") pod \"perf-node-gather-daemonset-nspfb\" (UID: \"7bd1b273-db16-460b-89a8-94c4ebc0404e\") " pod="openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb" Sep 08 02:36:51.747454 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:51.747440 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7bd1b273-db16-460b-89a8-94c4ebc0404e-lib-modules\") pod \"perf-node-gather-daemonset-nspfb\" (UID: \"7bd1b273-db16-460b-89a8-94c4ebc0404e\") " pod="openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb" Sep 08 02:36:51.747653 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:51.747638 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssgr6\" (UniqueName: \"kubernetes.io/projected/7bd1b273-db16-460b-89a8-94c4ebc0404e-kube-api-access-ssgr6\") pod \"perf-node-gather-daemonset-nspfb\" (UID: \"7bd1b273-db16-460b-89a8-94c4ebc0404e\") " pod="openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb" Sep 08 02:36:51.747746 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:51.747735 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7bd1b273-db16-460b-89a8-94c4ebc0404e-podres\") pod \"perf-node-gather-daemonset-nspfb\" (UID: \"7bd1b273-db16-460b-89a8-94c4ebc0404e\") " pod="openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb" Sep 08 02:36:51.848714 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:51.848663 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssgr6\" (UniqueName: \"kubernetes.io/projected/7bd1b273-db16-460b-89a8-94c4ebc0404e-kube-api-access-ssgr6\") pod \"perf-node-gather-daemonset-nspfb\" (UID: \"7bd1b273-db16-460b-89a8-94c4ebc0404e\") " pod="openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb" Sep 08 02:36:51.848893 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:51.848729 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7bd1b273-db16-460b-89a8-94c4ebc0404e-podres\") pod \"perf-node-gather-daemonset-nspfb\" (UID: \"7bd1b273-db16-460b-89a8-94c4ebc0404e\") " pod="openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb" Sep 08 02:36:51.848893 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:51.848801 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7bd1b273-db16-460b-89a8-94c4ebc0404e-proc\") pod \"perf-node-gather-daemonset-nspfb\" (UID: \"7bd1b273-db16-460b-89a8-94c4ebc0404e\") " pod="openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb" Sep 08 02:36:51.848893 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:51.848827 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7bd1b273-db16-460b-89a8-94c4ebc0404e-sys\") pod \"perf-node-gather-daemonset-nspfb\" (UID: \"7bd1b273-db16-460b-89a8-94c4ebc0404e\") " pod="openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb" Sep 08 02:36:51.849045 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:51.848914 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7bd1b273-db16-460b-89a8-94c4ebc0404e-lib-modules\") pod \"perf-node-gather-daemonset-nspfb\" (UID: \"7bd1b273-db16-460b-89a8-94c4ebc0404e\") " pod="openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb" Sep 08 02:36:51.849193 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:51.849175 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7bd1b273-db16-460b-89a8-94c4ebc0404e-lib-modules\") pod \"perf-node-gather-daemonset-nspfb\" (UID: \"7bd1b273-db16-460b-89a8-94c4ebc0404e\") " pod="openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb" Sep 08 02:36:51.849273 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:51.849255 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7bd1b273-db16-460b-89a8-94c4ebc0404e-proc\") pod \"perf-node-gather-daemonset-nspfb\" (UID: \"7bd1b273-db16-460b-89a8-94c4ebc0404e\") " pod="openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb" Sep 08 02:36:51.849400 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:51.849370 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7bd1b273-db16-460b-89a8-94c4ebc0404e-sys\") pod \"perf-node-gather-daemonset-nspfb\" (UID: \"7bd1b273-db16-460b-89a8-94c4ebc0404e\") " pod="openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb" Sep 08 02:36:51.849723 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:51.849699 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7bd1b273-db16-460b-89a8-94c4ebc0404e-podres\") pod \"perf-node-gather-daemonset-nspfb\" (UID: \"7bd1b273-db16-460b-89a8-94c4ebc0404e\") " pod="openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb" Sep 08 02:36:51.861222 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:51.861191 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssgr6\" (UniqueName: \"kubernetes.io/projected/7bd1b273-db16-460b-89a8-94c4ebc0404e-kube-api-access-ssgr6\") pod \"perf-node-gather-daemonset-nspfb\" (UID: \"7bd1b273-db16-460b-89a8-94c4ebc0404e\") " pod="openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb" Sep 08 02:36:52.023022 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:52.022934 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb" Sep 08 02:36:52.328477 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:52.328426 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb"] Sep 08 02:36:52.329809 ip-10-0-141-33 kubenswrapper[2560]: W0908 02:36:52.329780 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7bd1b273_db16_460b_89a8_94c4ebc0404e.slice/crio-ab5711a6a829521cecec701cdd898c3c6a7e20a50961d8a79c0ff390794267d9 WatchSource:0}: Error finding container ab5711a6a829521cecec701cdd898c3c6a7e20a50961d8a79c0ff390794267d9: Status 404 returned error can't find the container with id ab5711a6a829521cecec701cdd898c3c6a7e20a50961d8a79c0ff390794267d9 Sep 08 02:36:52.353788 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:52.353764 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-gc2gg_2d721641-4617-481e-8947-e5058100df47/download-server/0.log" Sep 08 02:36:52.728172 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:52.728129 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb" event={"ID":"7bd1b273-db16-460b-89a8-94c4ebc0404e","Type":"ContainerStarted","Data":"26740a14c56eb43c85c6177db1877f50283c70e160f9d37a13c8cd97c124d53a"} Sep 08 02:36:52.728741 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:52.728201 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb" event={"ID":"7bd1b273-db16-460b-89a8-94c4ebc0404e","Type":"ContainerStarted","Data":"ab5711a6a829521cecec701cdd898c3c6a7e20a50961d8a79c0ff390794267d9"} Sep 08 02:36:52.728741 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:52.728457 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb" Sep 08 02:36:52.748614 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:52.748551 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb" podStartSLOduration=1.748528154 podStartE2EDuration="1.748528154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 02:36:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 02:36:52.746978555 +0000 UTC m=+908.278623423" watchObservedRunningTime="2026-09-08 02:36:52.748528154 +0000 UTC m=+908.280173020" Sep 08 02:36:54.234311 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:54.234284 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qhkj2_382809a0-5fcd-4635-8ab5-f7e83672b1a5/dns/0.log" Sep 08 02:36:54.257444 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:54.257417 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qhkj2_382809a0-5fcd-4635-8ab5-f7e83672b1a5/kube-rbac-proxy/0.log" Sep 08 02:36:54.281322 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:54.281159 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-66vxb_185c152f-08c1-48bf-9d40-f61ee106ff12/dns-node-resolver/0.log" Sep 08 02:36:54.936443 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:54.936410 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vn7fg_6fe0dc65-7890-405f-9c55-755957407695/node-ca/0.log" Sep 08 02:36:56.000688 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:56.000656 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65f559df56-cqzn5_7b7b6a75-411c-41dd-b5fe-e61ff6172318/kube-auth-proxy/0.log" Sep 08 02:36:56.143729 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:56.143696 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-8jtjc_eedd08c8-e8e8-4464-9da0-c522a2d09a2b/payload-processing/0.log" Sep 08 02:36:56.174672 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:56.174642 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d6c5474d8-9wrc6_55e533a6-3267-4b11-966b-677e70829961/router/0.log" Sep 08 02:36:56.900867 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:56.900840 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p6qlw_aeddc1eb-1d10-4d91-a627-6d0cae0ad53b/serve-healthcheck-canary/0.log" Sep 08 02:36:57.401261 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:57.401223 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-2hccm_312a0520-e0c1-45b6-a9ac-e9f0b54df733/insights-operator/1.log" Sep 08 02:36:57.401261 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:57.401234 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-2hccm_312a0520-e0c1-45b6-a9ac-e9f0b54df733/insights-operator/0.log" Sep 08 02:36:57.497851 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:57.497823 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lg2vn_321393ae-48e7-4aab-b619-9f176576c695/kube-rbac-proxy/0.log" Sep 08 02:36:57.521134 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:57.521102 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lg2vn_321393ae-48e7-4aab-b619-9f176576c695/exporter/0.log" Sep 08 02:36:57.547110 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:57.547072 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lg2vn_321393ae-48e7-4aab-b619-9f176576c695/extractor/0.log" Sep 08 02:36:58.745828 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:58.745802 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jf4m6/perf-node-gather-daemonset-nspfb" Sep 08 02:36:59.960254 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:59.960228 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-5c89fb8b78-qjpw2_2414c1ef-e298-49fd-a86a-557585b4b622/manager/0.log" Sep 08 02:36:59.982535 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:36:59.982502 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-5c89fb8b78-qjpw2_2414c1ef-e298-49fd-a86a-557585b4b622/copy-manifests/0.log" Sep 08 02:37:01.300261 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:37:01.300228 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65f559df56-cqzn5_7b7b6a75-411c-41dd-b5fe-e61ff6172318/kube-auth-proxy/0.log" Sep 08 02:37:01.405588 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:37:01.405516 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-8jtjc_eedd08c8-e8e8-4464-9da0-c522a2d09a2b/payload-processing/0.log" Sep 08 02:37:01.438326 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:37:01.438294 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d6c5474d8-9wrc6_55e533a6-3267-4b11-966b-677e70829961/router/0.log" Sep 08 02:37:02.085261 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:37:02.085228 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-85d49875c6-xnn5f_c45c3758-5159-43b1-a8b0-58c14735ae03/manager/0.log" Sep 08 02:37:02.129871 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:37:02.129796 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-54765d59fd-rvkzh_dc55461f-bb4a-4cf8-aa2a-0f7ebd6a5435/manager/0.log" Sep 08 02:37:02.214516 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:37:02.214485 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-657f59599b-dpqc5_18bd4cc6-3225-4611-971f-611be27802ee/manager/0.log" Sep 08 02:37:02.235617 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:37:02.235585 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-7d6487fcc8-gqr6x_2c57646e-c736-4ca0-9f9a-8c271a1e8169/server/0.log" Sep 08 02:37:02.279727 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:37:02.279679 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-bb7b75cc4-g7dxh_6ba0375c-8d15-4d23-b19e-63c45bfa0e2a/manager/0.log" Sep 08 02:37:02.776979 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:37:02.776941 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65f559df56-cqzn5_7b7b6a75-411c-41dd-b5fe-e61ff6172318/kube-auth-proxy/0.log" Sep 08 02:37:02.874316 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:37:02.874283 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-8jtjc_eedd08c8-e8e8-4464-9da0-c522a2d09a2b/payload-processing/0.log" Sep 08 02:37:02.892163 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:37:02.892135 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d6c5474d8-9wrc6_55e533a6-3267-4b11-966b-677e70829961/router/0.log" Sep 08 02:37:07.234543 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:37:07.234513 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-tp6v9_d7b07bfb-8a8b-4404-a9c7-13762299a74f/migrator/0.log" Sep 08 02:37:07.257759 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:37:07.257733 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-tp6v9_d7b07bfb-8a8b-4404-a9c7-13762299a74f/graceful-termination/0.log" Sep 08 02:37:08.634861 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:37:08.634828 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-252ld_196615ff-9773-44ba-b5a7-a56229dafb40/kube-multus/0.log" Sep 08 02:37:08.691513 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:37:08.691479 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8mcwm_ab6018f8-3012-4ec8-959e-95479604e898/kube-multus-additional-cni-plugins/0.log" Sep 08 02:37:08.717554 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:37:08.717511 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8mcwm_ab6018f8-3012-4ec8-959e-95479604e898/egress-router-binary-copy/0.log" Sep 08 02:37:08.745570 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:37:08.745540 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8mcwm_ab6018f8-3012-4ec8-959e-95479604e898/cni-plugins/0.log" Sep 08 02:37:08.770112 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:37:08.770086 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8mcwm_ab6018f8-3012-4ec8-959e-95479604e898/bond-cni-plugin/0.log" Sep 08 02:37:08.797057 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:37:08.797029 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8mcwm_ab6018f8-3012-4ec8-959e-95479604e898/routeoverride-cni/0.log" Sep 08 02:37:08.821604 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:37:08.821574 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8mcwm_ab6018f8-3012-4ec8-959e-95479604e898/whereabouts-cni-bincopy/0.log" Sep 08 02:37:08.846686 ip-10-0-141-33 kubenswrapper[2560]: I0908 02:37:08.846658 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8mcwm_ab6018f8-3012-4ec8-959e-95479604e898/whereabouts-cni/0.log"