Aug 04 11:33:06.656173 ip-10-0-128-47 systemd[1]: Starting Kubernetes Kubelet... Aug 04 11:33:06.963147 ip-10-0-128-47 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 11:33:06.963147 ip-10-0-128-47 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Aug 04 11:33:06.963147 ip-10-0-128-47 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 11:33:06.963147 ip-10-0-128-47 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Aug 04 11:33:06.963744 ip-10-0-128-47 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 11:33:06.964608 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.964529 2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Aug 04 11:33:06.969528 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969507 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 11:33:06.969528 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969526 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 11:33:06.969528 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969531 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 11:33:06.969658 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969536 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 11:33:06.969658 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969541 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 11:33:06.969658 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969546 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 11:33:06.969658 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969550 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 11:33:06.969658 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969554 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 11:33:06.969658 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969559 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 11:33:06.969658 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969564 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 11:33:06.969658 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969568 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 11:33:06.969658 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969572 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 11:33:06.969658 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969576 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 11:33:06.969658 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969580 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 11:33:06.969658 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969586 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 11:33:06.969658 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969590 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 11:33:06.969658 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969594 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 11:33:06.969658 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969599 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 11:33:06.969658 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969603 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 11:33:06.969658 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969608 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 11:33:06.969658 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969612 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 11:33:06.969658 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969616 2570 feature_gate.go:328] unrecognized feature gate: Example Aug 04 11:33:06.970152 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969620 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 11:33:06.970152 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969624 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 11:33:06.970152 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969628 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 11:33:06.970152 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969635 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 11:33:06.970152 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969641 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 11:33:06.970152 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969646 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 11:33:06.970152 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969651 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 11:33:06.970152 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969655 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 11:33:06.970152 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969664 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 11:33:06.970152 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969669 2570 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 11:33:06.970152 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969674 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 11:33:06.970152 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969677 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 11:33:06.970152 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969680 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 11:33:06.970152 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969683 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 11:33:06.970152 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969685 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 11:33:06.970152 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969688 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 11:33:06.970152 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969690 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 11:33:06.970152 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969693 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 11:33:06.970152 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969695 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 11:33:06.970152 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969698 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 11:33:06.970645 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969700 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 11:33:06.970645 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969703 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 11:33:06.970645 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969705 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 11:33:06.970645 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969708 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 11:33:06.970645 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969711 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 11:33:06.970645 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969713 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 11:33:06.970645 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969716 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 11:33:06.970645 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969718 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 11:33:06.970645 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969722 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 11:33:06.970645 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969725 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 11:33:06.970645 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969728 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 11:33:06.970645 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969730 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 11:33:06.970645 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969732 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 11:33:06.970645 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969736 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 11:33:06.970645 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969738 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 11:33:06.970645 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969741 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 11:33:06.970645 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969744 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 11:33:06.970645 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969746 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 11:33:06.970645 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969748 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 11:33:06.970645 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969751 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 11:33:06.971149 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969754 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 11:33:06.971149 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969756 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 11:33:06.971149 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969759 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 11:33:06.971149 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969762 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 11:33:06.971149 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969765 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 11:33:06.971149 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969770 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 11:33:06.971149 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969773 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 11:33:06.971149 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969776 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 11:33:06.971149 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969778 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 11:33:06.971149 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969781 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 11:33:06.971149 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969784 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 11:33:06.971149 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969786 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 11:33:06.971149 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969789 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 11:33:06.971149 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969791 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 11:33:06.971149 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969794 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 11:33:06.971149 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969796 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 11:33:06.971149 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969799 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 11:33:06.971149 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969803 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 11:33:06.971149 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969805 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 11:33:06.971695 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969808 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 11:33:06.971695 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969811 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 11:33:06.971695 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969814 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 11:33:06.971695 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969816 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 11:33:06.971695 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969819 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 11:33:06.971695 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969822 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 11:33:06.971695 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969826 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 11:33:06.971695 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969828 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 11:33:06.971695 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969831 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 11:33:06.971695 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969833 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 11:33:06.971695 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969836 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 11:33:06.971695 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969839 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 11:33:06.971695 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969841 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 11:33:06.971695 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969844 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 11:33:06.971695 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969847 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 11:33:06.971695 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969851 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 11:33:06.971695 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969854 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 11:33:06.971695 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.969858 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 11:33:06.971695 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970566 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 11:33:06.972160 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970573 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 11:33:06.972160 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970578 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 11:33:06.972160 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970583 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 11:33:06.972160 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970587 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 11:33:06.972160 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970591 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 11:33:06.972160 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970594 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 11:33:06.972160 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970597 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 11:33:06.972160 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970600 2570 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 11:33:06.972160 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970603 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 11:33:06.972160 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970606 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 11:33:06.972160 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970609 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 11:33:06.972160 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970611 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 11:33:06.972160 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970614 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 11:33:06.972160 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970617 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 11:33:06.972160 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970619 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 11:33:06.972160 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970622 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 11:33:06.972160 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970625 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 11:33:06.972160 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970628 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 11:33:06.972160 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970631 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 11:33:06.972624 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970634 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 11:33:06.972624 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970636 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 11:33:06.972624 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970639 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 11:33:06.972624 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970642 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 11:33:06.972624 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970645 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 11:33:06.972624 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970649 2570 feature_gate.go:328] unrecognized feature gate: Example Aug 04 11:33:06.972624 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970651 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 11:33:06.972624 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970654 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 11:33:06.972624 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970657 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 11:33:06.972624 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970659 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 11:33:06.972624 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970662 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 11:33:06.972624 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970664 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 11:33:06.972624 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970667 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 11:33:06.972624 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970669 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 11:33:06.972624 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970671 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 11:33:06.972624 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970674 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 11:33:06.972624 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970676 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 11:33:06.972624 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970679 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 11:33:06.972624 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970681 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 11:33:06.972624 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970683 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 11:33:06.973127 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970686 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 11:33:06.973127 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970688 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 11:33:06.973127 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970691 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 11:33:06.973127 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970694 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 11:33:06.973127 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970697 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 11:33:06.973127 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970699 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 11:33:06.973127 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970702 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 11:33:06.973127 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970704 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 11:33:06.973127 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970708 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 11:33:06.973127 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970710 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 11:33:06.973127 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970714 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 11:33:06.973127 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970718 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 11:33:06.973127 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970721 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 11:33:06.973127 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970724 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 11:33:06.973127 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970727 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 11:33:06.973127 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970730 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 11:33:06.973127 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970732 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 11:33:06.973127 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970735 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 11:33:06.973127 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970738 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 11:33:06.973595 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970741 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 11:33:06.973595 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970743 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 11:33:06.973595 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970746 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 11:33:06.973595 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970748 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 11:33:06.973595 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970751 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 11:33:06.973595 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970753 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 11:33:06.973595 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970755 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 11:33:06.973595 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970758 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 11:33:06.973595 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970760 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 11:33:06.973595 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970763 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 11:33:06.973595 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970765 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 11:33:06.973595 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970767 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 11:33:06.973595 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970770 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 11:33:06.973595 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970773 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 11:33:06.973595 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970776 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 11:33:06.973595 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970778 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 11:33:06.973595 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970781 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 11:33:06.973595 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970784 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 11:33:06.973595 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970787 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 11:33:06.973595 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970790 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 11:33:06.974088 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970792 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 11:33:06.974088 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970795 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 11:33:06.974088 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970798 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 11:33:06.974088 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970801 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 11:33:06.974088 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970803 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 11:33:06.974088 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970807 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 11:33:06.974088 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970810 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 11:33:06.974088 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970812 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 11:33:06.974088 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970815 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 11:33:06.974088 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970817 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 11:33:06.974088 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970820 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 11:33:06.974088 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970822 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 11:33:06.974088 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970825 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 11:33:06.974088 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970827 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 11:33:06.974088 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970830 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 11:33:06.974088 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970833 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 11:33:06.974088 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970835 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 11:33:06.974088 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970838 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 11:33:06.974088 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970840 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 11:33:06.974088 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.970843 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 11:33:06.974607 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971267 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 11:33:06.974607 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971274 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 11:33:06.974607 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971277 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 11:33:06.974607 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971280 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 11:33:06.974607 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971283 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 11:33:06.974607 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971286 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 11:33:06.974607 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971288 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 11:33:06.974607 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971291 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 11:33:06.974607 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971295 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 11:33:06.974607 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971299 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 11:33:06.974607 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971302 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 11:33:06.974607 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971305 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 11:33:06.974607 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971307 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 11:33:06.974607 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971310 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 11:33:06.974607 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971312 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 11:33:06.974607 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971315 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 11:33:06.974607 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971318 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 11:33:06.974607 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971321 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 11:33:06.974607 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971323 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 11:33:06.974607 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971326 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 11:33:06.975107 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971329 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 11:33:06.975107 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971331 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 11:33:06.975107 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971333 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 11:33:06.975107 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971336 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 11:33:06.975107 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971339 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 11:33:06.975107 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971341 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 11:33:06.975107 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971344 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 11:33:06.975107 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971346 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 11:33:06.975107 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971349 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 11:33:06.975107 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971351 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 11:33:06.975107 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971354 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 11:33:06.975107 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971356 2570 feature_gate.go:328] unrecognized feature gate: Example Aug 04 11:33:06.975107 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971359 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 11:33:06.975107 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971361 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 11:33:06.975107 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971364 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 11:33:06.975107 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971366 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 11:33:06.975107 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971369 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 11:33:06.975107 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971371 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 11:33:06.975107 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971374 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 11:33:06.975107 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971376 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 11:33:06.975585 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971379 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 11:33:06.975585 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971385 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 11:33:06.975585 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971388 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 11:33:06.975585 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971393 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 11:33:06.975585 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971396 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 11:33:06.975585 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971399 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 11:33:06.975585 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971402 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 11:33:06.975585 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971405 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 11:33:06.975585 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971408 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 11:33:06.975585 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971411 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 11:33:06.975585 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971414 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 11:33:06.975585 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971417 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 11:33:06.975585 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971420 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 11:33:06.975585 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971423 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 11:33:06.975585 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971426 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 11:33:06.975585 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971428 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 11:33:06.975585 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971431 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 11:33:06.975585 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971433 2570 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 11:33:06.975585 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971436 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 11:33:06.975585 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971439 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 11:33:06.976120 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971442 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 11:33:06.976120 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971445 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 11:33:06.976120 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971447 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 11:33:06.976120 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971450 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 11:33:06.976120 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971452 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 11:33:06.976120 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971455 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 11:33:06.976120 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971457 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 11:33:06.976120 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971460 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 11:33:06.976120 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971464 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 11:33:06.976120 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971467 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 11:33:06.976120 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971470 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 11:33:06.976120 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971473 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 11:33:06.976120 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971476 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 11:33:06.976120 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971482 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 11:33:06.976120 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971486 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 11:33:06.976120 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971490 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 11:33:06.976120 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971493 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 11:33:06.976120 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971496 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 11:33:06.976120 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971498 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 11:33:06.976573 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971501 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 11:33:06.976573 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971504 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 11:33:06.976573 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971506 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 11:33:06.976573 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971510 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 11:33:06.976573 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971512 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 11:33:06.976573 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971515 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 11:33:06.976573 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971517 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 11:33:06.976573 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971520 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 11:33:06.976573 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971523 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 11:33:06.976573 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971525 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 11:33:06.976573 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971528 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 11:33:06.976573 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971530 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 11:33:06.976573 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971533 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 11:33:06.976573 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971536 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 11:33:06.976573 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971538 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 11:33:06.976573 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971541 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 11:33:06.976573 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971543 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 11:33:06.976573 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971545 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 11:33:06.976573 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971548 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 11:33:06.976573 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.971550 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971637 2570 flags.go:64] FLAG: --address="0.0.0.0" Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971647 2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971656 2570 flags.go:64] FLAG: --anonymous-auth="true" Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971663 2570 flags.go:64] FLAG: --application-metrics-count-limit="100" Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971669 2570 flags.go:64] FLAG: --authentication-token-webhook="false" Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971674 2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971679 2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971692 2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971696 2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971700 2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971703 2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971707 2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971710 2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971713 2570 flags.go:64] FLAG: --cgroup-root="" Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971716 2570 flags.go:64] FLAG: --cgroups-per-qos="true" Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971719 2570 flags.go:64] FLAG: --client-ca-file="" Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971727 2570 flags.go:64] FLAG: --cloud-provider="external" Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971730 2570 flags.go:64] FLAG: --cluster-dns="[]" Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971734 2570 flags.go:64] FLAG: --cluster-domain="" Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971736 2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971740 2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971743 2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971746 2570 flags.go:64] FLAG: --container-log-max-files="5" Aug 04 11:33:06.977058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971750 2570 flags.go:64] FLAG: --container-log-max-size="10Mi" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971753 2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971756 2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971759 2570 flags.go:64] FLAG: --containerd-namespace="k8s.io" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971762 2570 flags.go:64] FLAG: --contention-profiling="false" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971765 2570 flags.go:64] FLAG: --cpu-cfs-quota="true" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971768 2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971771 2570 flags.go:64] FLAG: --cpu-manager-policy="none" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971774 2570 flags.go:64] FLAG: --cpu-manager-policy-options="" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971778 2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971781 2570 flags.go:64] FLAG: --enable-controller-attach-detach="true" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971784 2570 flags.go:64] FLAG: --enable-debugging-handlers="true" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971787 2570 flags.go:64] FLAG: --enable-load-reader="false" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971790 2570 flags.go:64] FLAG: --enable-server="true" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971794 2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971802 2570 flags.go:64] FLAG: --event-burst="100" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971805 2570 flags.go:64] FLAG: --event-qps="50" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971808 2570 flags.go:64] FLAG: --event-storage-age-limit="default=0" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971811 2570 flags.go:64] FLAG: --event-storage-event-limit="default=0" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971814 2570 flags.go:64] FLAG: --eviction-hard="" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971817 2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971820 2570 flags.go:64] FLAG: --eviction-minimum-reclaim="" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971823 2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971826 2570 flags.go:64] FLAG: --eviction-soft="" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971829 2570 flags.go:64] FLAG: --eviction-soft-grace-period="" Aug 04 11:33:06.977668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971832 2570 flags.go:64] FLAG: --exit-on-lock-contention="false" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971840 2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971843 2570 flags.go:64] FLAG: --experimental-mounter-path="" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971846 2570 flags.go:64] FLAG: --fail-cgroupv1="false" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971848 2570 flags.go:64] FLAG: --fail-swap-on="true" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971851 2570 flags.go:64] FLAG: --feature-gates="" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971855 2570 flags.go:64] FLAG: --file-check-frequency="20s" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971858 2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971861 2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971864 2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971867 2570 flags.go:64] FLAG: --healthz-port="10248" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971870 2570 flags.go:64] FLAG: --help="false" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971873 2570 flags.go:64] FLAG: --hostname-override="ip-10-0-128-47.ec2.internal" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971876 2570 flags.go:64] FLAG: --housekeeping-interval="10s" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971879 2570 flags.go:64] FLAG: --http-check-frequency="20s" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971882 2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971885 2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971888 2570 flags.go:64] FLAG: --image-gc-high-threshold="85" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971891 2570 flags.go:64] FLAG: --image-gc-low-threshold="80" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971894 2570 flags.go:64] FLAG: --image-service-endpoint="" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971897 2570 flags.go:64] FLAG: --kernel-memcg-notification="false" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971914 2570 flags.go:64] FLAG: --kube-api-burst="100" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971919 2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971922 2570 flags.go:64] FLAG: --kube-api-qps="50" Aug 04 11:33:06.978304 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971925 2570 flags.go:64] FLAG: --kube-reserved="" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971928 2570 flags.go:64] FLAG: --kube-reserved-cgroup="" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971931 2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971934 2570 flags.go:64] FLAG: --kubelet-cgroups="" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971937 2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971940 2570 flags.go:64] FLAG: --lock-file="" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971943 2570 flags.go:64] FLAG: --log-cadvisor-usage="false" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971946 2570 flags.go:64] FLAG: --log-flush-frequency="5s" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971949 2570 flags.go:64] FLAG: --log-json-info-buffer-size="0" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971954 2570 flags.go:64] FLAG: --log-json-split-stream="false" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971963 2570 flags.go:64] FLAG: --log-text-info-buffer-size="0" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971966 2570 flags.go:64] FLAG: --log-text-split-stream="false" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971969 2570 flags.go:64] FLAG: --logging-format="text" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971971 2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971975 2570 flags.go:64] FLAG: --make-iptables-util-chains="true" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971978 2570 flags.go:64] FLAG: --manifest-url="" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971981 2570 flags.go:64] FLAG: --manifest-url-header="" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971985 2570 flags.go:64] FLAG: --max-housekeeping-interval="15s" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971988 2570 flags.go:64] FLAG: --max-open-files="1000000" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971992 2570 flags.go:64] FLAG: --max-pods="110" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971994 2570 flags.go:64] FLAG: --maximum-dead-containers="-1" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.971997 2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972000 2570 flags.go:64] FLAG: --memory-manager-policy="None" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972003 2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972006 2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Aug 04 11:33:06.978915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972009 2570 flags.go:64] FLAG: --node-ip="0.0.0.0" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972011 2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972018 2570 flags.go:64] FLAG: --node-status-max-images="50" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972021 2570 flags.go:64] FLAG: --node-status-update-frequency="10s" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972025 2570 flags.go:64] FLAG: --oom-score-adj="-999" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972030 2570 flags.go:64] FLAG: --pod-cidr="" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972032 2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972038 2570 flags.go:64] FLAG: --pod-manifest-path="" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972041 2570 flags.go:64] FLAG: --pod-max-pids="-1" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972044 2570 flags.go:64] FLAG: --pods-per-core="0" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972047 2570 flags.go:64] FLAG: --port="10250" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972050 2570 flags.go:64] FLAG: --protect-kernel-defaults="false" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972054 2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0701179f0031686b2" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972057 2570 flags.go:64] FLAG: --qos-reserved="" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972060 2570 flags.go:64] FLAG: --read-only-port="10255" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972063 2570 flags.go:64] FLAG: --register-node="true" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972065 2570 flags.go:64] FLAG: --register-with-taints="" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972069 2570 flags.go:64] FLAG: --registry-burst="10" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972073 2570 flags.go:64] FLAG: --registry-qps="5" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972075 2570 flags.go:64] FLAG: --reserved-cpus="" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972078 2570 flags.go:64] FLAG: --reserved-memory="" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972081 2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972084 2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972087 2570 flags.go:64] FLAG: --rotate-certificates="false" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972090 2570 flags.go:64] FLAG: --rotate-server-certificates="false" Aug 04 11:33:06.979525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972093 2570 flags.go:64] FLAG: --runonce="false" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972095 2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972099 2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972102 2570 flags.go:64] FLAG: --seccomp-default="false" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972104 2570 flags.go:64] FLAG: --serialize-image-pulls="true" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972107 2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972110 2570 flags.go:64] FLAG: --storage-driver-db="cadvisor" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972113 2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972116 2570 flags.go:64] FLAG: --storage-driver-password="root" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972119 2570 flags.go:64] FLAG: --storage-driver-secure="false" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972122 2570 flags.go:64] FLAG: --storage-driver-table="stats" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972126 2570 flags.go:64] FLAG: --storage-driver-user="root" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972129 2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972133 2570 flags.go:64] FLAG: --sync-frequency="1m0s" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972136 2570 flags.go:64] FLAG: --system-cgroups="" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972139 2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972144 2570 flags.go:64] FLAG: --system-reserved-cgroup="" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972147 2570 flags.go:64] FLAG: --tls-cert-file="" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972150 2570 flags.go:64] FLAG: --tls-cipher-suites="[]" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972156 2570 flags.go:64] FLAG: --tls-min-version="" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972158 2570 flags.go:64] FLAG: --tls-private-key-file="" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972161 2570 flags.go:64] FLAG: --topology-manager-policy="none" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972164 2570 flags.go:64] FLAG: --topology-manager-policy-options="" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972167 2570 flags.go:64] FLAG: --topology-manager-scope="container" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972170 2570 flags.go:64] FLAG: --v="2" Aug 04 11:33:06.980145 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972174 2570 flags.go:64] FLAG: --version="false" Aug 04 11:33:06.980742 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972179 2570 flags.go:64] FLAG: --vmodule="" Aug 04 11:33:06.980742 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972182 2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Aug 04 11:33:06.980742 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.972185 2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Aug 04 11:33:06.980742 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972295 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 11:33:06.980742 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972298 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 11:33:06.980742 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972301 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 11:33:06.980742 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972304 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 11:33:06.980742 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972307 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 11:33:06.980742 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972310 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 11:33:06.980742 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972313 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 11:33:06.980742 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972316 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 11:33:06.980742 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972319 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 11:33:06.980742 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972321 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 11:33:06.980742 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972324 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 11:33:06.980742 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972327 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 11:33:06.980742 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972329 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 11:33:06.980742 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972332 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 11:33:06.980742 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972336 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 11:33:06.980742 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972339 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 11:33:06.980742 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972342 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 11:33:06.981281 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972345 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 11:33:06.981281 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972347 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 11:33:06.981281 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972350 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 11:33:06.981281 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972352 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 11:33:06.981281 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972355 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 11:33:06.981281 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972357 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 11:33:06.981281 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972360 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 11:33:06.981281 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972362 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 11:33:06.981281 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972365 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 11:33:06.981281 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972367 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 11:33:06.981281 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972370 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 11:33:06.981281 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972373 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 11:33:06.981281 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972375 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 11:33:06.981281 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972385 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 11:33:06.981281 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972388 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 11:33:06.981281 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972390 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 11:33:06.981281 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972393 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 11:33:06.981281 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972395 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 11:33:06.981281 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972397 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 11:33:06.981281 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972400 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 11:33:06.981767 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972402 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 11:33:06.981767 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972405 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 11:33:06.981767 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972408 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 11:33:06.981767 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972410 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 11:33:06.981767 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972412 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 11:33:06.981767 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972415 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 11:33:06.981767 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972417 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 11:33:06.981767 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972420 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 11:33:06.981767 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972422 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 11:33:06.981767 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972427 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 11:33:06.981767 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972430 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 11:33:06.981767 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972433 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 11:33:06.981767 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972435 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 11:33:06.981767 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972438 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 11:33:06.981767 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972442 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 11:33:06.981767 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972445 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 11:33:06.981767 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972448 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 11:33:06.981767 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972451 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 11:33:06.981767 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972454 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 11:33:06.982246 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972457 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 11:33:06.982246 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972459 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 11:33:06.982246 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972462 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 11:33:06.982246 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972464 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 11:33:06.982246 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972466 2570 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 11:33:06.982246 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972469 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 11:33:06.982246 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972472 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 11:33:06.982246 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972476 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 11:33:06.982246 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972479 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 11:33:06.982246 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972482 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 11:33:06.982246 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972486 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 11:33:06.982246 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972489 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 11:33:06.982246 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972491 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 11:33:06.982246 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972494 2570 feature_gate.go:328] unrecognized feature gate: Example Aug 04 11:33:06.982246 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972496 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 11:33:06.982246 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972498 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 11:33:06.982246 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972501 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 11:33:06.982246 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972504 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 11:33:06.982246 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972506 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 11:33:06.982246 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972509 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 11:33:06.982741 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972511 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 11:33:06.982741 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972514 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 11:33:06.982741 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972517 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 11:33:06.982741 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972520 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 11:33:06.982741 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972523 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 11:33:06.982741 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972525 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 11:33:06.982741 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972529 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 11:33:06.982741 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972533 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 11:33:06.982741 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972536 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 11:33:06.982741 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972539 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 11:33:06.982741 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972541 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 11:33:06.982741 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972544 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 11:33:06.982741 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972547 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 11:33:06.982741 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972549 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 11:33:06.982741 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972552 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 11:33:06.982741 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972554 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 11:33:06.982741 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972557 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 11:33:06.982741 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972559 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 11:33:06.982741 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972562 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 11:33:06.982741 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972564 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 11:33:06.983231 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972567 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 11:33:06.983231 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972570 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 11:33:06.983231 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.972572 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 11:33:06.983231 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.973214 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 11:33:06.983231 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.979416 2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Aug 04 11:33:06.983231 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.979430 2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Aug 04 11:33:06.983231 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.979449 2570 watchdog_linux.go:95] "Systemd watchdog is not enabled" Aug 04 11:33:06.983231 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.979456 2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Aug 04 11:33:06.983231 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979509 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 11:33:06.983231 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979513 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 11:33:06.983231 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979517 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 11:33:06.983231 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979520 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 11:33:06.983231 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979522 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 11:33:06.983231 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979525 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 11:33:06.983588 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979528 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 11:33:06.983588 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979531 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 11:33:06.983588 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979534 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 11:33:06.983588 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979537 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 11:33:06.983588 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979540 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 11:33:06.983588 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979542 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 11:33:06.983588 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979545 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 11:33:06.983588 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979548 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 11:33:06.983588 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979551 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 11:33:06.983588 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979553 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 11:33:06.983588 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979556 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 11:33:06.983588 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979559 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 11:33:06.983588 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979562 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 11:33:06.983588 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979565 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 11:33:06.983588 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979567 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 11:33:06.983588 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979570 2570 feature_gate.go:328] unrecognized feature gate: Example Aug 04 11:33:06.983588 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979572 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 11:33:06.983588 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979575 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 11:33:06.983588 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979578 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 11:33:06.983588 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979581 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 11:33:06.984077 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979584 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 11:33:06.984077 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979587 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 11:33:06.984077 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979589 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 11:33:06.984077 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979593 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 11:33:06.984077 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979595 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 11:33:06.984077 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979598 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 11:33:06.984077 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979601 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 11:33:06.984077 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979603 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 11:33:06.984077 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979606 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 11:33:06.984077 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979609 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 11:33:06.984077 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979612 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 11:33:06.984077 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979614 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 11:33:06.984077 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979617 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 11:33:06.984077 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979619 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 11:33:06.984077 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979622 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 11:33:06.984077 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979624 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 11:33:06.984077 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979627 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 11:33:06.984077 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979630 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 11:33:06.984077 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979633 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 11:33:06.984077 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979635 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 11:33:06.984556 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979638 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 11:33:06.984556 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979640 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 11:33:06.984556 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979642 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 11:33:06.984556 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979646 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 11:33:06.984556 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979650 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 11:33:06.984556 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979653 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 11:33:06.984556 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979657 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 11:33:06.984556 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979660 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 11:33:06.984556 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979663 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 11:33:06.984556 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979666 2570 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 11:33:06.984556 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979669 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 11:33:06.984556 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979673 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 11:33:06.984556 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979678 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 11:33:06.984556 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979680 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 11:33:06.984556 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979683 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 11:33:06.984556 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979686 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 11:33:06.984556 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979689 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 11:33:06.984556 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979691 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 11:33:06.984556 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979694 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 11:33:06.985028 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979697 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 11:33:06.985028 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979699 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 11:33:06.985028 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979701 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 11:33:06.985028 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979704 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 11:33:06.985028 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979707 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 11:33:06.985028 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979709 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 11:33:06.985028 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979712 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 11:33:06.985028 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979715 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 11:33:06.985028 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979717 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 11:33:06.985028 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979720 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 11:33:06.985028 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979722 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 11:33:06.985028 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979725 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 11:33:06.985028 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979728 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 11:33:06.985028 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979730 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 11:33:06.985028 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979733 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 11:33:06.985028 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979736 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 11:33:06.985028 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979738 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 11:33:06.985028 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979741 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 11:33:06.985028 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979743 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 11:33:06.985504 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979746 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 11:33:06.985504 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979748 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 11:33:06.985504 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979751 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 11:33:06.985504 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979753 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 11:33:06.985504 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979756 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 11:33:06.985504 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979758 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 11:33:06.985504 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979761 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 11:33:06.985504 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979764 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 11:33:06.985504 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979767 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 11:33:06.985504 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979770 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 11:33:06.985504 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979773 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 11:33:06.985504 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979775 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 11:33:06.985504 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979778 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 11:33:06.985504 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979780 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 11:33:06.985504 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979783 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 11:33:06.985874 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.979788 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 11:33:06.985874 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979927 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 11:33:06.985874 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979934 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 11:33:06.985874 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979937 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 11:33:06.985874 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979941 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 11:33:06.985874 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979944 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 11:33:06.985874 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979946 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 11:33:06.985874 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979949 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 11:33:06.985874 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979952 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 11:33:06.985874 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979954 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 11:33:06.985874 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979957 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 11:33:06.985874 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979960 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 11:33:06.985874 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979963 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 11:33:06.985874 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979965 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 11:33:06.985874 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979968 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 11:33:06.986268 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979970 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 11:33:06.986268 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979972 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 11:33:06.986268 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979975 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 11:33:06.986268 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979977 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 11:33:06.986268 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979980 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 11:33:06.986268 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979983 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 11:33:06.986268 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979985 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 11:33:06.986268 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979988 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 11:33:06.986268 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979991 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 11:33:06.986268 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979994 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 11:33:06.986268 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.979997 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 11:33:06.986268 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980000 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 11:33:06.986268 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980003 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 11:33:06.986268 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980006 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 11:33:06.986268 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980009 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 11:33:06.986268 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980011 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 11:33:06.986268 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980014 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 11:33:06.986268 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980017 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 11:33:06.986268 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980019 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 11:33:06.986268 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980022 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 11:33:06.986756 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980024 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 11:33:06.986756 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980027 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 11:33:06.986756 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980029 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 11:33:06.986756 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980032 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 11:33:06.986756 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980034 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 11:33:06.986756 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980037 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 11:33:06.986756 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980039 2570 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 11:33:06.986756 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980042 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 11:33:06.986756 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980044 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 11:33:06.986756 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980047 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 11:33:06.986756 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980050 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 11:33:06.986756 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980052 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 11:33:06.986756 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980054 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 11:33:06.986756 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980057 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 11:33:06.986756 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980059 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 11:33:06.986756 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980062 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 11:33:06.986756 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980064 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 11:33:06.986756 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980067 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 11:33:06.986756 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980069 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 11:33:06.986756 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980072 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 11:33:06.987296 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980074 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 11:33:06.987296 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980077 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 11:33:06.987296 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980080 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 11:33:06.987296 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980083 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 11:33:06.987296 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980086 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 11:33:06.987296 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980089 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 11:33:06.987296 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980092 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 11:33:06.987296 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980095 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 11:33:06.987296 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980098 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 11:33:06.987296 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980100 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 11:33:06.987296 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980103 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 11:33:06.987296 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980105 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 11:33:06.987296 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980108 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 11:33:06.987296 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980110 2570 feature_gate.go:328] unrecognized feature gate: Example Aug 04 11:33:06.987296 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980113 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 11:33:06.987296 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980116 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 11:33:06.987296 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980120 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 11:33:06.987296 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980122 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 11:33:06.987296 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980125 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 11:33:06.987763 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980128 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 11:33:06.987763 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980130 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 11:33:06.987763 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980133 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 11:33:06.987763 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980135 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 11:33:06.987763 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980138 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 11:33:06.987763 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980140 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 11:33:06.987763 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980143 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 11:33:06.987763 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980146 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 11:33:06.987763 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980148 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 11:33:06.987763 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980151 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 11:33:06.987763 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980154 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 11:33:06.987763 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980157 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 11:33:06.987763 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980159 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 11:33:06.987763 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980162 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 11:33:06.987763 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980164 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 11:33:06.987763 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980167 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 11:33:06.987763 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980170 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 11:33:06.987763 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980173 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 11:33:06.987763 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980175 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 11:33:06.987763 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980178 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 11:33:06.988323 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980182 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 11:33:06.988323 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980185 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 11:33:06.988323 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980188 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 11:33:06.988323 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980191 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 11:33:06.988323 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980194 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 11:33:06.988323 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:06.980197 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 11:33:06.988323 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.980201 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 11:33:06.988323 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.980291 2570 server.go:962] "Client rotation is on, will bootstrap in background" Aug 04 11:33:06.988323 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.982315 2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Aug 04 11:33:06.988323 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.983368 2570 server.go:1019] "Starting client certificate rotation" Aug 04 11:33:06.988323 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.983460 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 04 11:33:06.988323 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.983494 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 04 11:33:06.988323 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.985394 2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 04 11:33:06.988649 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.986513 2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 04 11:33:06.995520 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.995501 2570 log.go:25] "Validated CRI v1 runtime API" Aug 04 11:33:06.997438 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.997420 2570 log.go:25] "Validated CRI v1 image API" Aug 04 11:33:06.998223 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:06.998205 2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Aug 04 11:33:07.001053 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.001033 2570 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9a37febf-21fc-43db-ad03-3b7f93f767fa:/dev/nvme0n1p4 d64c340e-66cb-4516-a27c-79f49014527b:/dev/nvme0n1p3] Aug 04 11:33:07.001114 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.001055 2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Aug 04 11:33:07.008003 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.007857 2570 manager.go:216] Machine: {Timestamp:2026-08-04 11:33:07.006422492 +0000 UTC m=+0.256685678 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100427 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bacc129357881aa0eac25a9cbdcfd SystemUUID:ec2bacc1-2935-7881-aa0e-ac25a9cbdcfd BootID:9a369e40-c06b-43a7-84e2-5ee151f09422 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6f:10:46:d5:fb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6f:10:46:d5:fb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:32:e1:44:82:35:62 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Aug 04 11:33:07.008096 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.008003 2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Aug 04 11:33:07.008130 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.008094 2570 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Aug 04 11:33:07.010429 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.010407 2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Aug 04 11:33:07.010565 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.010432 2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-47.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Aug 04 11:33:07.011075 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.011065 2570 topology_manager.go:138] "Creating topology manager with none policy" Aug 04 11:33:07.011111 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.011076 2570 container_manager_linux.go:309] "Creating device plugin manager" Aug 04 11:33:07.011111 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.011084 2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 04 11:33:07.011636 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.011626 2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 04 11:33:07.012433 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.012421 2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Aug 04 11:33:07.016603 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.016591 2570 state_mem.go:36] "Initialized new in-memory state store" Aug 04 11:33:07.016737 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.016727 2570 server.go:1240] "Using root directory" path="/var/lib/kubelet" Aug 04 11:33:07.017346 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.017337 2570 kubelet.go:486] "Attempting to sync node with API server" Aug 04 11:33:07.017383 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.017349 2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Aug 04 11:33:07.017383 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.017365 2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Aug 04 11:33:07.017383 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.017374 2570 kubelet.go:398] "Adding apiserver pod source" Aug 04 11:33:07.017383 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.017383 2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Aug 04 11:33:07.017756 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.017744 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 04 11:33:07.017799 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.017764 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 04 11:33:07.019575 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.019558 2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Aug 04 11:33:07.021127 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.021109 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 04 11:33:07.021252 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.021238 2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Aug 04 11:33:07.021298 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.021275 2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Aug 04 11:33:07.022294 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.022279 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Aug 04 11:33:07.022335 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.022301 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Aug 04 11:33:07.022335 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.022308 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Aug 04 11:33:07.022335 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.022314 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Aug 04 11:33:07.022335 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.022322 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Aug 04 11:33:07.022335 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.022328 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Aug 04 11:33:07.022335 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.022336 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Aug 04 11:33:07.022502 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.022342 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Aug 04 11:33:07.022502 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.022348 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Aug 04 11:33:07.022502 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.022354 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Aug 04 11:33:07.022502 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.022369 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Aug 04 11:33:07.022502 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.022374 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Aug 04 11:33:07.022946 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.022937 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Aug 04 11:33:07.022985 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.022947 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Aug 04 11:33:07.026334 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.026314 2570 server.go:1268] "Started kubelet" Aug 04 11:33:07.027012 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.026961 2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Aug 04 11:33:07.027100 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.027039 2570 server_v1.go:49] "podresources" method="list" useActivePods=true Aug 04 11:33:07.027954 ip-10-0-128-47 systemd[1]: Started Kubernetes Kubelet. Aug 04 11:33:07.031465 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.027958 2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Aug 04 11:33:07.031465 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.028150 2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Aug 04 11:33:07.031465 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.030788 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-47.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Aug 04 11:33:07.031465 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.030859 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Aug 04 11:33:07.031465 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.030960 2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-47.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Aug 04 11:33:07.031465 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.031020 2570 server.go:310] "Adding debug handlers to kubelet server" Aug 04 11:33:07.032719 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.032699 2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Aug 04 11:33:07.034551 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.034535 2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Aug 04 11:33:07.034657 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.034556 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Aug 04 11:33:07.034711 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.034676 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-47.ec2.internal\" not found" Aug 04 11:33:07.035056 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.035042 2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Aug 04 11:33:07.035156 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.035045 2570 volume_manager.go:311] "The desired_state_of_world populator starts" Aug 04 11:33:07.035156 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.035073 2570 volume_manager.go:313] "Starting Kubelet Volume Manager" Aug 04 11:33:07.035156 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.035148 2570 reconstruct.go:97] "Volume reconstruction finished" Aug 04 11:33:07.035156 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.035157 2570 reconciler.go:29] "Reconciler: start to sync state" Aug 04 11:33:07.035679 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.035660 2570 factory.go:55] Registering systemd factory Aug 04 11:33:07.035679 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.035681 2570 factory.go:223] Registration of the systemd container factory successfully Aug 04 11:33:07.037565 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.037548 2570 factory.go:153] Registering CRI-O factory Aug 04 11:33:07.037565 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.037565 2570 factory.go:223] Registration of the crio container factory successfully Aug 04 11:33:07.037685 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.037606 2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Aug 04 11:33:07.037685 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.037634 2570 factory.go:103] Registering Raw factory Aug 04 11:33:07.037685 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.037647 2570 manager.go:1196] Started watching for new ooms in manager Aug 04 11:33:07.038048 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.038028 2570 manager.go:349] Starting recovery of all containers Aug 04 11:33:07.045831 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.044981 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-47.ec2.internal.18c896eab340db37 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-47.ec2.internal,UID:ip-10-0-128-47.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-47.ec2.internal,},FirstTimestamp:2026-08-04 11:33:07.026287415 +0000 UTC m=+0.276550602,LastTimestamp:2026-08-04 11:33:07.026287415 +0000 UTC m=+0.276550602,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-47.ec2.internal,}" Aug 04 11:33:07.045955 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.045930 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Aug 04 11:33:07.048247 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.048227 2570 manager.go:354] Recovery completed Aug 04 11:33:07.049808 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.049786 2570 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Aug 04 11:33:07.053211 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.053197 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 11:33:07.054215 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.054194 2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-47.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Aug 04 11:33:07.057056 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.057022 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientMemory" Aug 04 11:33:07.057131 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.057075 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 11:33:07.057131 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.057092 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientPID" Aug 04 11:33:07.057674 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.057658 2570 cpu_manager.go:222] "Starting CPU manager" policy="none" Aug 04 11:33:07.057756 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.057673 2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Aug 04 11:33:07.057756 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.057698 2570 state_mem.go:36] "Initialized new in-memory state store" Aug 04 11:33:07.057756 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.057663 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 11:33:07.058444 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.058430 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientMemory" Aug 04 11:33:07.058513 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.058455 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 11:33:07.058513 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.058465 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientPID" Aug 04 11:33:07.059821 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.059810 2570 policy_none.go:49] "None policy: Start" Aug 04 11:33:07.059865 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.059825 2570 memory_manager.go:187] "Starting memorymanager" policy="None" Aug 04 11:33:07.059865 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.059836 2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Aug 04 11:33:07.061764 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.061754 2570 policy_none.go:47] "Start" Aug 04 11:33:07.067654 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.067585 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-47.ec2.internal.18c896eab51656d6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-47.ec2.internal,UID:ip-10-0-128-47.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-47.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-47.ec2.internal,},FirstTimestamp:2026-08-04 11:33:07.057055446 +0000 UTC m=+0.307318852,LastTimestamp:2026-08-04 11:33:07.057055446 +0000 UTC m=+0.307318852,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-47.ec2.internal,}" Aug 04 11:33:07.072879 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.071995 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-47.ec2.internal.18c896eab516c297 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-47.ec2.internal,UID:ip-10-0-128-47.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-47.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-47.ec2.internal,},FirstTimestamp:2026-08-04 11:33:07.057083031 +0000 UTC m=+0.307346221,LastTimestamp:2026-08-04 11:33:07.057083031 +0000 UTC m=+0.307346221,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-47.ec2.internal,}" Aug 04 11:33:07.076540 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.076327 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-47.ec2.internal.18c896eab516ff7d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-47.ec2.internal,UID:ip-10-0-128-47.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-47.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-47.ec2.internal,},FirstTimestamp:2026-08-04 11:33:07.057098621 +0000 UTC m=+0.307361805,LastTimestamp:2026-08-04 11:33:07.057098621 +0000 UTC m=+0.307361805,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-47.ec2.internal,}" Aug 04 11:33:07.084058 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.083999 2570 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-47.ec2.internal.18c896eab51656d6\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-47.ec2.internal.18c896eab51656d6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-47.ec2.internal,UID:ip-10-0-128-47.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-47.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-47.ec2.internal,},FirstTimestamp:2026-08-04 11:33:07.057055446 +0000 UTC m=+0.307318852,LastTimestamp:2026-08-04 11:33:07.058443789 +0000 UTC m=+0.308706973,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-47.ec2.internal,}" Aug 04 11:33:07.087457 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.087442 2570 manager.go:337] "Starting Device Plugin manager" Aug 04 11:33:07.087544 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.087477 2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Aug 04 11:33:07.087544 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.087490 2570 server.go:87] "Starting device plugin registration server" Aug 04 11:33:07.087721 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.087708 2570 eviction_manager.go:189] "Eviction manager: starting control loop" Aug 04 11:33:07.087796 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.087727 2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Aug 04 11:33:07.087876 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.087861 2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Aug 04 11:33:07.087981 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.087966 2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Aug 04 11:33:07.088032 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.087981 2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Aug 04 11:33:07.088415 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.088389 2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Aug 04 11:33:07.088478 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.088432 2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-47.ec2.internal\" not found" Aug 04 11:33:07.094323 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.094263 2570 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-47.ec2.internal.18c896eab516c297\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-47.ec2.internal.18c896eab516c297 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-47.ec2.internal,UID:ip-10-0-128-47.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-47.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-47.ec2.internal,},FirstTimestamp:2026-08-04 11:33:07.057083031 +0000 UTC m=+0.307346221,LastTimestamp:2026-08-04 11:33:07.058459423 +0000 UTC m=+0.308722606,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-47.ec2.internal,}" Aug 04 11:33:07.102256 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.102236 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-w5dsn" Aug 04 11:33:07.104877 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.104823 2570 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-47.ec2.internal.18c896eab516ff7d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-47.ec2.internal.18c896eab516ff7d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-47.ec2.internal,UID:ip-10-0-128-47.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-47.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-47.ec2.internal,},FirstTimestamp:2026-08-04 11:33:07.057098621 +0000 UTC m=+0.307361805,LastTimestamp:2026-08-04 11:33:07.058468899 +0000 UTC m=+0.308732083,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-47.ec2.internal,}" Aug 04 11:33:07.108606 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.108591 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-w5dsn" Aug 04 11:33:07.113084 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.113030 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-47.ec2.internal.18c896eab70463c2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-47.ec2.internal,UID:ip-10-0-128-47.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-128-47.ec2.internal,},FirstTimestamp:2026-08-04 11:33:07.089433538 +0000 UTC m=+0.339696713,LastTimestamp:2026-08-04 11:33:07.089433538 +0000 UTC m=+0.339696713,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-47.ec2.internal,}" Aug 04 11:33:07.181961 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.181939 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Aug 04 11:33:07.184069 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.183020 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Aug 04 11:33:07.184069 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.183062 2570 status_manager.go:244] "Starting to sync pod status with apiserver" Aug 04 11:33:07.184069 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.183086 2570 kubelet.go:2448] "Starting kubelet main sync loop" Aug 04 11:33:07.184069 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.183124 2570 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Aug 04 11:33:07.187822 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.187807 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 11:33:07.188461 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.188448 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientMemory" Aug 04 11:33:07.188534 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.188472 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 11:33:07.188534 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.188483 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientPID" Aug 04 11:33:07.188534 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.188505 2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-47.ec2.internal" Aug 04 11:33:07.194541 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.194524 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 11:33:07.202323 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.202307 2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-47.ec2.internal" Aug 04 11:33:07.202397 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.202328 2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-47.ec2.internal\": node \"ip-10-0-128-47.ec2.internal\" not found" Aug 04 11:33:07.216727 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.216677 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-47.ec2.internal\" not found" Aug 04 11:33:07.284090 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.284069 2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-47.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal"] Aug 04 11:33:07.284169 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.284123 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 11:33:07.284955 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.284940 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientMemory" Aug 04 11:33:07.285028 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.284968 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 11:33:07.285028 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.284978 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientPID" Aug 04 11:33:07.287244 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.287232 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 11:33:07.287356 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.287339 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-47.ec2.internal" Aug 04 11:33:07.287401 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.287370 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 11:33:07.287986 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.287967 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientMemory" Aug 04 11:33:07.288086 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.287997 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 11:33:07.288086 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.288010 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientPID" Aug 04 11:33:07.288086 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.287972 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientMemory" Aug 04 11:33:07.288086 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.288051 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 11:33:07.288086 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.288062 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientPID" Aug 04 11:33:07.290256 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.290241 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal" Aug 04 11:33:07.290339 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.290271 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 11:33:07.290939 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.290922 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientMemory" Aug 04 11:33:07.291039 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.290953 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 11:33:07.291039 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.290968 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientPID" Aug 04 11:33:07.313427 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.313405 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-47.ec2.internal\" not found" node="ip-10-0-128-47.ec2.internal" Aug 04 11:33:07.317380 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.317368 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-47.ec2.internal\" not found" Aug 04 11:33:07.317688 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.317677 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-47.ec2.internal\" not found" node="ip-10-0-128-47.ec2.internal" Aug 04 11:33:07.337812 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.337794 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/138335873249eb34a707d460b9840757-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal\" (UID: \"138335873249eb34a707d460b9840757\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal" Aug 04 11:33:07.337863 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.337817 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b3c19a22d5a3d4dd837bfebae044979e-config\") pod \"kube-apiserver-proxy-ip-10-0-128-47.ec2.internal\" (UID: \"b3c19a22d5a3d4dd837bfebae044979e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-47.ec2.internal" Aug 04 11:33:07.337863 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.337852 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/138335873249eb34a707d460b9840757-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal\" (UID: \"138335873249eb34a707d460b9840757\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal" Aug 04 11:33:07.418279 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:07.418253 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-47.ec2.internal\" not found" Aug 04 11:33:07.438859 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.438838 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b3c19a22d5a3d4dd837bfebae044979e-config\") pod \"kube-apiserver-proxy-ip-10-0-128-47.ec2.internal\" (UID: \"b3c19a22d5a3d4dd837bfebae044979e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-47.ec2.internal" Aug 04 11:33:07.438949 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.438864 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/138335873249eb34a707d460b9840757-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal\" (UID: \"138335873249eb34a707d460b9840757\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal" Aug 04 11:33:07.438949 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.438880 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/138335873249eb34a707d460b9840757-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal\" (UID: \"138335873249eb34a707d460b9840757\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal" Aug 04 11:33:07.438949 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.438933 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/138335873249eb34a707d460b9840757-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal\" (UID: \"138335873249eb34a707d460b9840757\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal" Aug 04 11:33:07.439060 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.438957 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b3c19a22d5a3d4dd837bfebae044979e-config\") pod \"kube-apiserver-proxy-ip-10-0-128-47.ec2.internal\" (UID: \"b3c19a22d5a3d4dd837bfebae044979e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-47.ec2.internal" Aug 04 11:33:07.439060 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:07.438957 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/138335873249eb34a707d460b9840757-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal\" (UID: \"138335873249eb34a707d460b9840757\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal" Aug 04 11:33:08.524732 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:08.524657 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-47.ec2.internal\" not found" Aug 04 11:33:08.622227 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:08.622187 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-47.ec2.internal" Aug 04 11:33:08.625304 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:08.625286 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-47.ec2.internal\" not found" Aug 04 11:33:08.626104 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:08.626093 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal" Aug 04 11:33:08.726216 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:08.726185 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-47.ec2.internal\" not found" Aug 04 11:33:08.826777 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:08.826711 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-47.ec2.internal\" not found" Aug 04 11:33:08.927187 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:08.927162 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-47.ec2.internal\" not found" Aug 04 11:33:08.988633 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:08.988605 2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Aug 04 11:33:08.989022 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:08.988737 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Aug 04 11:33:09.028060 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:09.028026 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-47.ec2.internal\" not found" Aug 04 11:33:09.079359 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:09.079310 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 11:33:09.081727 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:09.081711 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientMemory" Aug 04 11:33:09.081832 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:09.081744 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 11:33:09.081832 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:09.081759 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientPID" Aug 04 11:33:09.118998 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:09.118962 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-03 11:28:08 +0000 UTC" deadline="2028-04-26 21:21:56.470782448 +0000 UTC" Aug 04 11:33:09.118998 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:09.118996 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15153h48m47.351790224s" Aug 04 11:33:09.129117 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:09.129096 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-47.ec2.internal\" not found" Aug 04 11:33:09.212555 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:09.212516 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3c19a22d5a3d4dd837bfebae044979e.slice/crio-6b2225b20d6ba002dee1a86fb17f49d6a595bda1bc6f7f1e1747075421352f06 WatchSource:0}: Error finding container 6b2225b20d6ba002dee1a86fb17f49d6a595bda1bc6f7f1e1747075421352f06: Status 404 returned error can't find the container with id 6b2225b20d6ba002dee1a86fb17f49d6a595bda1bc6f7f1e1747075421352f06 Aug 04 11:33:09.213424 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:09.213401 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod138335873249eb34a707d460b9840757.slice/crio-35ab4100f6d295c1b5a720d3293aac8c7b2f970318a39412591ede3129166979 WatchSource:0}: Error finding container 35ab4100f6d295c1b5a720d3293aac8c7b2f970318a39412591ede3129166979: Status 404 returned error can't find the container with id 35ab4100f6d295c1b5a720d3293aac8c7b2f970318a39412591ede3129166979 Aug 04 11:33:09.218107 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:09.218094 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 11:33:09.230176 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:09.230147 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-47.ec2.internal\" not found" Aug 04 11:33:09.270931 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:09.270912 2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 11:33:09.340513 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:09.340457 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-47.ec2.internal" Aug 04 11:33:09.356981 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:09.356958 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 04 11:33:09.358174 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:09.358162 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal" Aug 04 11:33:09.366730 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:09.366712 2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 11:33:09.373504 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:09.373489 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 04 11:33:09.408612 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:09.408590 2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 11:33:10.024904 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.024847 2570 apiserver.go:52] "Watching apiserver" Aug 04 11:33:10.034430 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.034385 2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Aug 04 11:33:10.035526 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.035374 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-hbfpg","openshift-image-registry/node-ca-5z7cf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal","openshift-multus/multus-n4x7h","openshift-multus/network-metrics-daemon-kt4m7","kube-system/konnectivity-agent-4wnrp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv","openshift-cluster-node-tuning-operator/tuned-rjrxh","openshift-multus/multus-additional-cni-plugins-t6mz2","openshift-network-diagnostics/network-check-target-42v7s","openshift-network-operator/iptables-alerter-zlp8p","openshift-ovn-kubernetes/ovnkube-node-88j8c","kube-system/global-pull-secret-syncer-lcqpb","kube-system/kube-apiserver-proxy-ip-10-0-128-47.ec2.internal"] Aug 04 11:33:10.038251 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.038227 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" Aug 04 11:33:10.044088 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.041355 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Aug 04 11:33:10.044088 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.041907 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Aug 04 11:33:10.044088 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.042009 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Aug 04 11:33:10.044088 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.042092 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sb42h\"" Aug 04 11:33:10.044088 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.042184 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Aug 04 11:33:10.044088 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.042593 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:10.044088 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:10.042671 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lcqpb" podUID="e3158842-36a1-4213-af82-c6db89481715" Aug 04 11:33:10.044088 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.043051 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.046059 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.045006 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.046059 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.045388 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Aug 04 11:33:10.046059 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.045707 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Aug 04 11:33:10.046059 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.045913 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hp5mm\"" Aug 04 11:33:10.048370 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.047412 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Aug 04 11:33:10.048370 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.047803 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8r8kt\"" Aug 04 11:33:10.048370 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.047973 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Aug 04 11:33:10.048370 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.048114 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:10.048370 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:10.048167 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4m7" podUID="fe775771-24c4-46a8-b53a-4322c76ec540" Aug 04 11:33:10.048370 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.048232 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Aug 04 11:33:10.048770 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.048750 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Aug 04 11:33:10.049689 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.049283 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4wnrp" Aug 04 11:33:10.052223 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.051645 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.052223 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.051760 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Aug 04 11:33:10.052666 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.052556 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nsnmf\"" Aug 04 11:33:10.052666 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.052589 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5526e5d1-5421-4f6a-9b97-aa45435fece2-etc-tuned\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.052666 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.052632 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j27kj\" (UniqueName: \"kubernetes.io/projected/5526e5d1-5421-4f6a-9b97-aa45435fece2-kube-api-access-j27kj\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.052666 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.052657 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-os-release\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.052896 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.052693 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-host-var-lib-cni-multus\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.052896 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.052717 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvs9g\" (UniqueName: \"kubernetes.io/projected/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-kube-api-access-lvs9g\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.052896 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.052742 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e3158842-36a1-4213-af82-c6db89481715-dbus\") pod \"global-pull-secret-syncer-lcqpb\" (UID: \"e3158842-36a1-4213-af82-c6db89481715\") " pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:10.052896 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.052785 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-sys\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.052896 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.052829 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-multus-conf-dir\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.052896 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.052873 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bf58b5f4-9fac-4635-badb-1f2b39a971e4-device-dir\") pod \"aws-ebs-csi-driver-node-kdkpv\" (UID: \"bf58b5f4-9fac-4635-badb-1f2b39a971e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" Aug 04 11:33:10.053180 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.052915 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e3158842-36a1-4213-af82-c6db89481715-kubelet-config\") pod \"global-pull-secret-syncer-lcqpb\" (UID: \"e3158842-36a1-4213-af82-c6db89481715\") " pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:10.053180 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.052945 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-cni-binary-copy\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.053180 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.052967 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-multus-daemon-config\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.053180 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.052992 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-etc-kubernetes\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.053180 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053033 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bf58b5f4-9fac-4635-badb-1f2b39a971e4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kdkpv\" (UID: \"bf58b5f4-9fac-4635-badb-1f2b39a971e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" Aug 04 11:33:10.053180 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053055 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-etc-sysconfig\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.053180 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053077 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-lib-modules\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.053180 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053099 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-host-run-netns\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.053180 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053120 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bf58b5f4-9fac-4635-badb-1f2b39a971e4-socket-dir\") pod \"aws-ebs-csi-driver-node-kdkpv\" (UID: \"bf58b5f4-9fac-4635-badb-1f2b39a971e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" Aug 04 11:33:10.053180 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053145 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bf58b5f4-9fac-4635-badb-1f2b39a971e4-registration-dir\") pod \"aws-ebs-csi-driver-node-kdkpv\" (UID: \"bf58b5f4-9fac-4635-badb-1f2b39a971e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" Aug 04 11:33:10.053180 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053169 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bf58b5f4-9fac-4635-badb-1f2b39a971e4-sys-fs\") pod \"aws-ebs-csi-driver-node-kdkpv\" (UID: \"bf58b5f4-9fac-4635-badb-1f2b39a971e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" Aug 04 11:33:10.054830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053192 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lscp\" (UniqueName: \"kubernetes.io/projected/bf58b5f4-9fac-4635-badb-1f2b39a971e4-kube-api-access-9lscp\") pod \"aws-ebs-csi-driver-node-kdkpv\" (UID: \"bf58b5f4-9fac-4635-badb-1f2b39a971e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" Aug 04 11:33:10.054830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053225 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret\") pod \"global-pull-secret-syncer-lcqpb\" (UID: \"e3158842-36a1-4213-af82-c6db89481715\") " pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:10.054830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053248 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-etc-kubernetes\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.054830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053272 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-etc-systemd\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.054830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053292 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-host\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.054830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053320 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-var-lib-kubelet\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.054830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053347 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-multus-cni-dir\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.054830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053379 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-cnibin\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.054830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053421 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-host-var-lib-cni-bin\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.054830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053465 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-host-run-multus-certs\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.054830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053493 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-etc-sysctl-d\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.054830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053551 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-etc-sysctl-conf\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.054830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053574 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-run\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.054830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053603 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5526e5d1-5421-4f6a-9b97-aa45435fece2-tmp\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.054830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053627 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-host-run-k8s-cni-cncf-io\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.054830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053648 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-hostroot\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.054830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053684 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bf58b5f4-9fac-4635-badb-1f2b39a971e4-etc-selinux\") pod \"aws-ebs-csi-driver-node-kdkpv\" (UID: \"bf58b5f4-9fac-4635-badb-1f2b39a971e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" Aug 04 11:33:10.055392 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053719 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-etc-modprobe-d\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.055392 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053742 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-system-cni-dir\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.055392 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053766 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-multus-socket-dir-parent\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.055392 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053787 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-host-var-lib-kubelet\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.055392 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.053989 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Aug 04 11:33:10.055392 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.054049 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hbfpg" Aug 04 11:33:10.055392 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.054354 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nf4q8\"" Aug 04 11:33:10.055392 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.054629 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Aug 04 11:33:10.055392 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.054939 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Aug 04 11:33:10.057749 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.056058 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 04 11:33:10.057749 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.056353 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5z7cf" Aug 04 11:33:10.057749 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.056493 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Aug 04 11:33:10.057749 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.057018 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Aug 04 11:33:10.057749 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.057067 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hpwf7\"" Aug 04 11:33:10.058958 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.058937 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:10.059048 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:10.059002 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42v7s" podUID="f1f75655-126a-410b-ab43-16e96cc67d05" Aug 04 11:33:10.059106 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.059095 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Aug 04 11:33:10.061106 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.060591 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sq4gc\"" Aug 04 11:33:10.061106 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.060607 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Aug 04 11:33:10.061106 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.060634 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Aug 04 11:33:10.062095 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.061380 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zlp8p" Aug 04 11:33:10.065599 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.065513 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rzxnk\"" Aug 04 11:33:10.065696 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.065608 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.065752 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.065706 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Aug 04 11:33:10.065798 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.065745 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Aug 04 11:33:10.066092 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.065610 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Aug 04 11:33:10.068422 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.068277 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Aug 04 11:33:10.069293 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.068803 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Aug 04 11:33:10.069293 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.068822 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Aug 04 11:33:10.069293 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.068829 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Aug 04 11:33:10.069293 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.069126 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Aug 04 11:33:10.070057 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.070031 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5bck5\"" Aug 04 11:33:10.070251 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.070236 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Aug 04 11:33:10.096354 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.096332 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qwdll" Aug 04 11:33:10.105878 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.105833 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qwdll" Aug 04 11:33:10.142462 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.142420 2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Aug 04 11:33:10.144139 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.144119 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 11:33:10.153998 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.153973 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5526e5d1-5421-4f6a-9b97-aa45435fece2-tmp\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.154117 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154010 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-host-run-k8s-cni-cncf-io\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.154117 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154037 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-hostroot\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.154117 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154077 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c1cf79fd-d20b-4d9c-8b7d-e4eefe3e83c6-host-slash\") pod \"iptables-alerter-zlp8p\" (UID: \"c1cf79fd-d20b-4d9c-8b7d-e4eefe3e83c6\") " pod="openshift-network-operator/iptables-alerter-zlp8p" Aug 04 11:33:10.154117 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154101 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-var-lib-openvswitch\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.154117 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154104 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-host-run-k8s-cni-cncf-io\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.154281 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154138 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-hostroot\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.154281 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154179 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-host-run-ovn-kubernetes\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.154281 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154198 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-etc-modprobe-d\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.154281 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154213 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-system-cni-dir\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.154281 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154229 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-multus-socket-dir-parent\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.154281 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154244 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-host-var-lib-kubelet\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.154281 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154261 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57fbz\" (UniqueName: \"kubernetes.io/projected/ad3565b1-c324-4d62-88d6-274e04aaf220-kube-api-access-57fbz\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.154281 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154276 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-node-log\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.154689 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154291 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9dafc945-e2f2-4b29-8b6d-b27765cc5e6e-konnectivity-ca\") pod \"konnectivity-agent-4wnrp\" (UID: \"9dafc945-e2f2-4b29-8b6d-b27765cc5e6e\") " pod="kube-system/konnectivity-agent-4wnrp" Aug 04 11:33:10.154689 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154307 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5526e5d1-5421-4f6a-9b97-aa45435fece2-etc-tuned\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.154689 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154322 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-os-release\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.154689 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154323 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-host-var-lib-kubelet\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.154689 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154335 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-etc-openvswitch\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.154689 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154333 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-etc-modprobe-d\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.154689 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154343 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-system-cni-dir\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.154689 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154286 2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Aug 04 11:33:10.154689 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154385 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-run-ovn\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.154689 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154401 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-os-release\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.154689 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154416 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e3158842-36a1-4213-af82-c6db89481715-dbus\") pod \"global-pull-secret-syncer-lcqpb\" (UID: \"e3158842-36a1-4213-af82-c6db89481715\") " pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:10.154689 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154442 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-multus-conf-dir\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.154689 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154485 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad3565b1-c324-4d62-88d6-274e04aaf220-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.154689 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154508 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2e8c00dc-adfc-4152-901e-ce48741e273c-env-overrides\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.154689 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154529 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-multus-socket-dir-parent\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.154689 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154563 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-multus-conf-dir\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.154689 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154586 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e3158842-36a1-4213-af82-c6db89481715-dbus\") pod \"global-pull-secret-syncer-lcqpb\" (UID: \"e3158842-36a1-4213-af82-c6db89481715\") " pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:10.154689 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154604 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs\") pod \"network-metrics-daemon-kt4m7\" (UID: \"fe775771-24c4-46a8-b53a-4322c76ec540\") " pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:10.155558 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154638 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wq2c\" (UniqueName: \"kubernetes.io/projected/8f2acb16-9405-4d91-853d-6a54e7a03a59-kube-api-access-4wq2c\") pod \"node-ca-5z7cf\" (UID: \"8f2acb16-9405-4d91-853d-6a54e7a03a59\") " pod="openshift-image-registry/node-ca-5z7cf" Aug 04 11:33:10.155558 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154667 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-multus-daemon-config\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.155558 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154692 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-etc-kubernetes\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.155558 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154717 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ad3565b1-c324-4d62-88d6-274e04aaf220-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.155558 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154729 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-etc-kubernetes\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.155558 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154746 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.155558 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154771 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bf58b5f4-9fac-4635-badb-1f2b39a971e4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kdkpv\" (UID: \"bf58b5f4-9fac-4635-badb-1f2b39a971e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" Aug 04 11:33:10.155558 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154793 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-etc-sysconfig\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.155558 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154819 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bf58b5f4-9fac-4635-badb-1f2b39a971e4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kdkpv\" (UID: \"bf58b5f4-9fac-4635-badb-1f2b39a971e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" Aug 04 11:33:10.155558 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154832 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-etc-sysconfig\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.155558 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154849 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-lib-modules\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.155558 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154901 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad3565b1-c324-4d62-88d6-274e04aaf220-system-cni-dir\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.155558 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154926 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-host-slash\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.155558 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154948 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwjpc\" (UniqueName: \"kubernetes.io/projected/fe775771-24c4-46a8-b53a-4322c76ec540-kube-api-access-zwjpc\") pod \"network-metrics-daemon-kt4m7\" (UID: \"fe775771-24c4-46a8-b53a-4322c76ec540\") " pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:10.155558 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154970 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ad3565b1-c324-4d62-88d6-274e04aaf220-os-release\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.155558 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154995 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bf58b5f4-9fac-4635-badb-1f2b39a971e4-socket-dir\") pod \"aws-ebs-csi-driver-node-kdkpv\" (UID: \"bf58b5f4-9fac-4635-badb-1f2b39a971e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" Aug 04 11:33:10.155558 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.154996 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-lib-modules\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.156365 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155026 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-etc-kubernetes\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.156365 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155051 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-host\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.156365 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155074 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc62k\" (UniqueName: \"kubernetes.io/projected/2e8c00dc-adfc-4152-901e-ce48741e273c-kube-api-access-rc62k\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.156365 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155098 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46hhc\" (UniqueName: \"kubernetes.io/projected/f8dbbbc0-29a4-4849-9700-f9f8ebb310d8-kube-api-access-46hhc\") pod \"node-resolver-hbfpg\" (UID: \"f8dbbbc0-29a4-4849-9700-f9f8ebb310d8\") " pod="openshift-dns/node-resolver-hbfpg" Aug 04 11:33:10.156365 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155133 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-etc-kubernetes\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.156365 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155140 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8f2acb16-9405-4d91-853d-6a54e7a03a59-serviceca\") pod \"node-ca-5z7cf\" (UID: \"8f2acb16-9405-4d91-853d-6a54e7a03a59\") " pod="openshift-image-registry/node-ca-5z7cf" Aug 04 11:33:10.156365 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155113 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bf58b5f4-9fac-4635-badb-1f2b39a971e4-socket-dir\") pod \"aws-ebs-csi-driver-node-kdkpv\" (UID: \"bf58b5f4-9fac-4635-badb-1f2b39a971e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" Aug 04 11:33:10.156365 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155151 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-host\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.156365 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155184 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-var-lib-kubelet\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.156365 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155206 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-host-var-lib-cni-bin\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.156365 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155228 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2e8c00dc-adfc-4152-901e-ce48741e273c-ovnkube-script-lib\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.156365 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155248 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-etc-sysctl-conf\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.156365 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155259 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-host-var-lib-cni-bin\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.156365 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155263 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-var-lib-kubelet\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.156365 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155269 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-run\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.156365 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155310 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-run-systemd\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.156365 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155354 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-run\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.157179 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155464 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-etc-sysctl-conf\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.157179 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155461 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9dafc945-e2f2-4b29-8b6d-b27765cc5e6e-agent-certs\") pod \"konnectivity-agent-4wnrp\" (UID: \"9dafc945-e2f2-4b29-8b6d-b27765cc5e6e\") " pod="kube-system/konnectivity-agent-4wnrp" Aug 04 11:33:10.157179 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155497 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bf58b5f4-9fac-4635-badb-1f2b39a971e4-etc-selinux\") pod \"aws-ebs-csi-driver-node-kdkpv\" (UID: \"bf58b5f4-9fac-4635-badb-1f2b39a971e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" Aug 04 11:33:10.157179 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155526 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c1cf79fd-d20b-4d9c-8b7d-e4eefe3e83c6-iptables-alerter-script\") pod \"iptables-alerter-zlp8p\" (UID: \"c1cf79fd-d20b-4d9c-8b7d-e4eefe3e83c6\") " pod="openshift-network-operator/iptables-alerter-zlp8p" Aug 04 11:33:10.157179 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155595 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bf58b5f4-9fac-4635-badb-1f2b39a971e4-etc-selinux\") pod \"aws-ebs-csi-driver-node-kdkpv\" (UID: \"bf58b5f4-9fac-4635-badb-1f2b39a971e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" Aug 04 11:33:10.157179 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155619 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2e8c00dc-adfc-4152-901e-ce48741e273c-ovnkube-config\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.157179 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155640 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f8dbbbc0-29a4-4849-9700-f9f8ebb310d8-hosts-file\") pod \"node-resolver-hbfpg\" (UID: \"f8dbbbc0-29a4-4849-9700-f9f8ebb310d8\") " pod="openshift-dns/node-resolver-hbfpg" Aug 04 11:33:10.157179 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155660 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j27kj\" (UniqueName: \"kubernetes.io/projected/5526e5d1-5421-4f6a-9b97-aa45435fece2-kube-api-access-j27kj\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.157179 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155678 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-host-var-lib-cni-multus\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.157179 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155696 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvs9g\" (UniqueName: \"kubernetes.io/projected/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-kube-api-access-lvs9g\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.157179 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155716 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad3565b1-c324-4d62-88d6-274e04aaf220-cni-binary-copy\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.157179 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155715 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-host-var-lib-cni-multus\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.157179 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155730 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4w2l4\" (UniqueName: \"kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4\") pod \"network-check-target-42v7s\" (UID: \"f1f75655-126a-410b-ab43-16e96cc67d05\") " pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:10.157179 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155743 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-run-openvswitch\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.157179 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155757 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-host-cni-netd\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.157179 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155775 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2e8c00dc-adfc-4152-901e-ce48741e273c-ovn-node-metrics-cert\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.157179 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155793 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-sys\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.158006 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155811 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ad3565b1-c324-4d62-88d6-274e04aaf220-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.158006 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155824 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-systemd-units\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.158006 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155823 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-multus-daemon-config\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.158006 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155837 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-host-run-netns\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.158006 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155855 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bf58b5f4-9fac-4635-badb-1f2b39a971e4-device-dir\") pod \"aws-ebs-csi-driver-node-kdkpv\" (UID: \"bf58b5f4-9fac-4635-badb-1f2b39a971e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" Aug 04 11:33:10.158006 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155881 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e3158842-36a1-4213-af82-c6db89481715-kubelet-config\") pod \"global-pull-secret-syncer-lcqpb\" (UID: \"e3158842-36a1-4213-af82-c6db89481715\") " pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:10.158006 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155906 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-cni-binary-copy\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.158006 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155918 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-sys\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.158006 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155959 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-host-kubelet\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.158006 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.155990 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bf58b5f4-9fac-4635-badb-1f2b39a971e4-device-dir\") pod \"aws-ebs-csi-driver-node-kdkpv\" (UID: \"bf58b5f4-9fac-4635-badb-1f2b39a971e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" Aug 04 11:33:10.158006 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156002 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-host-run-netns\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.158006 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156031 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e3158842-36a1-4213-af82-c6db89481715-kubelet-config\") pod \"global-pull-secret-syncer-lcqpb\" (UID: \"e3158842-36a1-4213-af82-c6db89481715\") " pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:10.158006 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156073 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-cnibin\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.158006 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156102 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-host-run-netns\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.158006 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156147 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad3565b1-c324-4d62-88d6-274e04aaf220-cnibin\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.158006 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156174 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xfqt\" (UniqueName: \"kubernetes.io/projected/c1cf79fd-d20b-4d9c-8b7d-e4eefe3e83c6-kube-api-access-8xfqt\") pod \"iptables-alerter-zlp8p\" (UID: \"c1cf79fd-d20b-4d9c-8b7d-e4eefe3e83c6\") " pod="openshift-network-operator/iptables-alerter-zlp8p" Aug 04 11:33:10.158006 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156216 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-cnibin\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.158827 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156244 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bf58b5f4-9fac-4635-badb-1f2b39a971e4-registration-dir\") pod \"aws-ebs-csi-driver-node-kdkpv\" (UID: \"bf58b5f4-9fac-4635-badb-1f2b39a971e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" Aug 04 11:33:10.158827 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156271 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bf58b5f4-9fac-4635-badb-1f2b39a971e4-sys-fs\") pod \"aws-ebs-csi-driver-node-kdkpv\" (UID: \"bf58b5f4-9fac-4635-badb-1f2b39a971e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" Aug 04 11:33:10.158827 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156320 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bf58b5f4-9fac-4635-badb-1f2b39a971e4-registration-dir\") pod \"aws-ebs-csi-driver-node-kdkpv\" (UID: \"bf58b5f4-9fac-4635-badb-1f2b39a971e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" Aug 04 11:33:10.158827 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156327 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lscp\" (UniqueName: \"kubernetes.io/projected/bf58b5f4-9fac-4635-badb-1f2b39a971e4-kube-api-access-9lscp\") pod \"aws-ebs-csi-driver-node-kdkpv\" (UID: \"bf58b5f4-9fac-4635-badb-1f2b39a971e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" Aug 04 11:33:10.158827 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156344 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-cni-binary-copy\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.158827 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156362 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret\") pod \"global-pull-secret-syncer-lcqpb\" (UID: \"e3158842-36a1-4213-af82-c6db89481715\") " pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:10.158827 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156362 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bf58b5f4-9fac-4635-badb-1f2b39a971e4-sys-fs\") pod \"aws-ebs-csi-driver-node-kdkpv\" (UID: \"bf58b5f4-9fac-4635-badb-1f2b39a971e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" Aug 04 11:33:10.158827 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156480 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-etc-systemd\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.158827 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:10.156521 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 11:33:10.158827 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156536 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-multus-cni-dir\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.158827 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:10.156587 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret podName:e3158842-36a1-4213-af82-c6db89481715 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:10.65656758 +0000 UTC m=+2.901287123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret") pod "global-pull-secret-syncer-lcqpb" (UID: "e3158842-36a1-4213-af82-c6db89481715") : object "kube-system"/"original-pull-secret" not registered Aug 04 11:33:10.158827 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156610 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-host-run-multus-certs\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.158827 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156632 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-multus-cni-dir\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.158827 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156589 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-etc-systemd\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.158827 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156639 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-log-socket\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.158827 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156669 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-host-cni-bin\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.158827 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156683 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-host-run-multus-certs\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.159590 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156720 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8dbbbc0-29a4-4849-9700-f9f8ebb310d8-tmp-dir\") pod \"node-resolver-hbfpg\" (UID: \"f8dbbbc0-29a4-4849-9700-f9f8ebb310d8\") " pod="openshift-dns/node-resolver-hbfpg" Aug 04 11:33:10.159590 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156757 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f2acb16-9405-4d91-853d-6a54e7a03a59-host\") pod \"node-ca-5z7cf\" (UID: \"8f2acb16-9405-4d91-853d-6a54e7a03a59\") " pod="openshift-image-registry/node-ca-5z7cf" Aug 04 11:33:10.159590 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156804 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-etc-sysctl-d\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.159590 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.156934 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5526e5d1-5421-4f6a-9b97-aa45435fece2-etc-sysctl-d\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.159590 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.158625 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5526e5d1-5421-4f6a-9b97-aa45435fece2-etc-tuned\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.159590 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.158743 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5526e5d1-5421-4f6a-9b97-aa45435fece2-tmp\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.166531 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.166400 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j27kj\" (UniqueName: \"kubernetes.io/projected/5526e5d1-5421-4f6a-9b97-aa45435fece2-kube-api-access-j27kj\") pod \"tuned-rjrxh\" (UID: \"5526e5d1-5421-4f6a-9b97-aa45435fece2\") " pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.167013 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.166970 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvs9g\" (UniqueName: \"kubernetes.io/projected/b8a85cde-1d4e-4c4b-926f-fdec33a673a6-kube-api-access-lvs9g\") pod \"multus-n4x7h\" (UID: \"b8a85cde-1d4e-4c4b-926f-fdec33a673a6\") " pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.167726 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.167704 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lscp\" (UniqueName: \"kubernetes.io/projected/bf58b5f4-9fac-4635-badb-1f2b39a971e4-kube-api-access-9lscp\") pod \"aws-ebs-csi-driver-node-kdkpv\" (UID: \"bf58b5f4-9fac-4635-badb-1f2b39a971e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" Aug 04 11:33:10.193533 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.193430 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal" event={"ID":"138335873249eb34a707d460b9840757","Type":"ContainerStarted","Data":"35ab4100f6d295c1b5a720d3293aac8c7b2f970318a39412591ede3129166979"} Aug 04 11:33:10.195196 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.195172 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-47.ec2.internal" event={"ID":"b3c19a22d5a3d4dd837bfebae044979e","Type":"ContainerStarted","Data":"6b2225b20d6ba002dee1a86fb17f49d6a595bda1bc6f7f1e1747075421352f06"} Aug 04 11:33:10.257044 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257013 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46hhc\" (UniqueName: \"kubernetes.io/projected/f8dbbbc0-29a4-4849-9700-f9f8ebb310d8-kube-api-access-46hhc\") pod \"node-resolver-hbfpg\" (UID: \"f8dbbbc0-29a4-4849-9700-f9f8ebb310d8\") " pod="openshift-dns/node-resolver-hbfpg" Aug 04 11:33:10.257044 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257046 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8f2acb16-9405-4d91-853d-6a54e7a03a59-serviceca\") pod \"node-ca-5z7cf\" (UID: \"8f2acb16-9405-4d91-853d-6a54e7a03a59\") " pod="openshift-image-registry/node-ca-5z7cf" Aug 04 11:33:10.257250 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257064 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2e8c00dc-adfc-4152-901e-ce48741e273c-ovnkube-script-lib\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.257250 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257166 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-run-systemd\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.257250 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257188 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9dafc945-e2f2-4b29-8b6d-b27765cc5e6e-agent-certs\") pod \"konnectivity-agent-4wnrp\" (UID: \"9dafc945-e2f2-4b29-8b6d-b27765cc5e6e\") " pod="kube-system/konnectivity-agent-4wnrp" Aug 04 11:33:10.257250 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257211 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c1cf79fd-d20b-4d9c-8b7d-e4eefe3e83c6-iptables-alerter-script\") pod \"iptables-alerter-zlp8p\" (UID: \"c1cf79fd-d20b-4d9c-8b7d-e4eefe3e83c6\") " pod="openshift-network-operator/iptables-alerter-zlp8p" Aug 04 11:33:10.257250 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257232 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2e8c00dc-adfc-4152-901e-ce48741e273c-ovnkube-config\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.257250 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257252 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f8dbbbc0-29a4-4849-9700-f9f8ebb310d8-hosts-file\") pod \"node-resolver-hbfpg\" (UID: \"f8dbbbc0-29a4-4849-9700-f9f8ebb310d8\") " pod="openshift-dns/node-resolver-hbfpg" Aug 04 11:33:10.257471 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257270 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad3565b1-c324-4d62-88d6-274e04aaf220-cni-binary-copy\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.257471 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257285 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w2l4\" (UniqueName: \"kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4\") pod \"network-check-target-42v7s\" (UID: \"f1f75655-126a-410b-ab43-16e96cc67d05\") " pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:10.257471 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257300 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-run-openvswitch\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.257471 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257315 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-host-cni-netd\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.257471 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257343 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2e8c00dc-adfc-4152-901e-ce48741e273c-ovn-node-metrics-cert\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.257471 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257371 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ad3565b1-c324-4d62-88d6-274e04aaf220-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.257471 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257397 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-systemd-units\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.257471 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257433 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-host-run-netns\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.257471 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257467 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-host-kubelet\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.257785 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257484 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad3565b1-c324-4d62-88d6-274e04aaf220-cnibin\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.257785 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257498 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xfqt\" (UniqueName: \"kubernetes.io/projected/c1cf79fd-d20b-4d9c-8b7d-e4eefe3e83c6-kube-api-access-8xfqt\") pod \"iptables-alerter-zlp8p\" (UID: \"c1cf79fd-d20b-4d9c-8b7d-e4eefe3e83c6\") " pod="openshift-network-operator/iptables-alerter-zlp8p" Aug 04 11:33:10.257785 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257526 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-log-socket\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.257785 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257552 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-host-cni-bin\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.257785 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257575 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8dbbbc0-29a4-4849-9700-f9f8ebb310d8-tmp-dir\") pod \"node-resolver-hbfpg\" (UID: \"f8dbbbc0-29a4-4849-9700-f9f8ebb310d8\") " pod="openshift-dns/node-resolver-hbfpg" Aug 04 11:33:10.257785 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257584 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8f2acb16-9405-4d91-853d-6a54e7a03a59-serviceca\") pod \"node-ca-5z7cf\" (UID: \"8f2acb16-9405-4d91-853d-6a54e7a03a59\") " pod="openshift-image-registry/node-ca-5z7cf" Aug 04 11:33:10.257785 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257599 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f2acb16-9405-4d91-853d-6a54e7a03a59-host\") pod \"node-ca-5z7cf\" (UID: \"8f2acb16-9405-4d91-853d-6a54e7a03a59\") " pod="openshift-image-registry/node-ca-5z7cf" Aug 04 11:33:10.257785 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257634 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c1cf79fd-d20b-4d9c-8b7d-e4eefe3e83c6-host-slash\") pod \"iptables-alerter-zlp8p\" (UID: \"c1cf79fd-d20b-4d9c-8b7d-e4eefe3e83c6\") " pod="openshift-network-operator/iptables-alerter-zlp8p" Aug 04 11:33:10.257785 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257644 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-run-systemd\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.257785 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257650 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-var-lib-openvswitch\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.257785 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257666 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-host-run-ovn-kubernetes\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.257785 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257723 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c1cf79fd-d20b-4d9c-8b7d-e4eefe3e83c6-iptables-alerter-script\") pod \"iptables-alerter-zlp8p\" (UID: \"c1cf79fd-d20b-4d9c-8b7d-e4eefe3e83c6\") " pod="openshift-network-operator/iptables-alerter-zlp8p" Aug 04 11:33:10.257785 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.257755 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-var-lib-openvswitch\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.258268 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258051 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-systemd-units\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.258268 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258180 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ad3565b1-c324-4d62-88d6-274e04aaf220-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.258268 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258221 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2e8c00dc-adfc-4152-901e-ce48741e273c-ovnkube-script-lib\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.258268 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258238 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f8dbbbc0-29a4-4849-9700-f9f8ebb310d8-hosts-file\") pod \"node-resolver-hbfpg\" (UID: \"f8dbbbc0-29a4-4849-9700-f9f8ebb310d8\") " pod="openshift-dns/node-resolver-hbfpg" Aug 04 11:33:10.258268 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258261 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-run-openvswitch\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.258455 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258269 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-host-run-netns\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.258455 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258295 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-host-run-ovn-kubernetes\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.258455 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258300 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-host-cni-bin\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.258455 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258309 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57fbz\" (UniqueName: \"kubernetes.io/projected/ad3565b1-c324-4d62-88d6-274e04aaf220-kube-api-access-57fbz\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.258455 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258349 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-node-log\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.258455 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258379 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9dafc945-e2f2-4b29-8b6d-b27765cc5e6e-konnectivity-ca\") pod \"konnectivity-agent-4wnrp\" (UID: \"9dafc945-e2f2-4b29-8b6d-b27765cc5e6e\") " pod="kube-system/konnectivity-agent-4wnrp" Aug 04 11:33:10.258455 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258432 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-etc-openvswitch\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.258783 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258472 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-run-ovn\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.258783 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258502 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad3565b1-c324-4d62-88d6-274e04aaf220-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.258783 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258524 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8f2acb16-9405-4d91-853d-6a54e7a03a59-host\") pod \"node-ca-5z7cf\" (UID: \"8f2acb16-9405-4d91-853d-6a54e7a03a59\") " pod="openshift-image-registry/node-ca-5z7cf" Aug 04 11:33:10.258783 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258528 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2e8c00dc-adfc-4152-901e-ce48741e273c-env-overrides\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.258783 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258565 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs\") pod \"network-metrics-daemon-kt4m7\" (UID: \"fe775771-24c4-46a8-b53a-4322c76ec540\") " pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:10.258783 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258596 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wq2c\" (UniqueName: \"kubernetes.io/projected/8f2acb16-9405-4d91-853d-6a54e7a03a59-kube-api-access-4wq2c\") pod \"node-ca-5z7cf\" (UID: \"8f2acb16-9405-4d91-853d-6a54e7a03a59\") " pod="openshift-image-registry/node-ca-5z7cf" Aug 04 11:33:10.258783 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258630 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ad3565b1-c324-4d62-88d6-274e04aaf220-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.258783 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:10.258640 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 11:33:10.258783 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258657 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.258783 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:10.258703 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs podName:fe775771-24c4-46a8-b53a-4322c76ec540 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:10.758682152 +0000 UTC m=+3.003401687 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs") pod "network-metrics-daemon-kt4m7" (UID: "fe775771-24c4-46a8-b53a-4322c76ec540") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 11:33:10.258783 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258707 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8dbbbc0-29a4-4849-9700-f9f8ebb310d8-tmp-dir\") pod \"node-resolver-hbfpg\" (UID: \"f8dbbbc0-29a4-4849-9700-f9f8ebb310d8\") " pod="openshift-dns/node-resolver-hbfpg" Aug 04 11:33:10.258783 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258723 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad3565b1-c324-4d62-88d6-274e04aaf220-system-cni-dir\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.258783 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258732 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2e8c00dc-adfc-4152-901e-ce48741e273c-ovnkube-config\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.258783 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258749 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-host-slash\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.258783 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258761 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-host-cni-netd\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.258783 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258775 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwjpc\" (UniqueName: \"kubernetes.io/projected/fe775771-24c4-46a8-b53a-4322c76ec540-kube-api-access-zwjpc\") pod \"network-metrics-daemon-kt4m7\" (UID: \"fe775771-24c4-46a8-b53a-4322c76ec540\") " pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:10.259517 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258802 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ad3565b1-c324-4d62-88d6-274e04aaf220-os-release\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.259517 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258832 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc62k\" (UniqueName: \"kubernetes.io/projected/2e8c00dc-adfc-4152-901e-ce48741e273c-kube-api-access-rc62k\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.259517 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258916 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad3565b1-c324-4d62-88d6-274e04aaf220-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.259517 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.258972 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-host-kubelet\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.259517 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.259015 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-node-log\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.259517 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.259039 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad3565b1-c324-4d62-88d6-274e04aaf220-system-cni-dir\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.259517 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.259067 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.259517 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.259109 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ad3565b1-c324-4d62-88d6-274e04aaf220-os-release\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.259517 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.259146 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2e8c00dc-adfc-4152-901e-ce48741e273c-env-overrides\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.259517 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.259193 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad3565b1-c324-4d62-88d6-274e04aaf220-cnibin\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.259517 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.259287 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c1cf79fd-d20b-4d9c-8b7d-e4eefe3e83c6-host-slash\") pod \"iptables-alerter-zlp8p\" (UID: \"c1cf79fd-d20b-4d9c-8b7d-e4eefe3e83c6\") " pod="openshift-network-operator/iptables-alerter-zlp8p" Aug 04 11:33:10.259517 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.259409 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-etc-openvswitch\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.259517 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.259467 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-run-ovn\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.259517 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.259500 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-host-slash\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.259998 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.259577 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ad3565b1-c324-4d62-88d6-274e04aaf220-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.259998 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.259602 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2e8c00dc-adfc-4152-901e-ce48741e273c-log-socket\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.259998 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.259648 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad3565b1-c324-4d62-88d6-274e04aaf220-cni-binary-copy\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.260231 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.260209 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9dafc945-e2f2-4b29-8b6d-b27765cc5e6e-konnectivity-ca\") pod \"konnectivity-agent-4wnrp\" (UID: \"9dafc945-e2f2-4b29-8b6d-b27765cc5e6e\") " pod="kube-system/konnectivity-agent-4wnrp" Aug 04 11:33:10.260398 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.260378 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9dafc945-e2f2-4b29-8b6d-b27765cc5e6e-agent-certs\") pod \"konnectivity-agent-4wnrp\" (UID: \"9dafc945-e2f2-4b29-8b6d-b27765cc5e6e\") " pod="kube-system/konnectivity-agent-4wnrp" Aug 04 11:33:10.261577 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.261555 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2e8c00dc-adfc-4152-901e-ce48741e273c-ovn-node-metrics-cert\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.266792 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.266769 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46hhc\" (UniqueName: \"kubernetes.io/projected/f8dbbbc0-29a4-4849-9700-f9f8ebb310d8-kube-api-access-46hhc\") pod \"node-resolver-hbfpg\" (UID: \"f8dbbbc0-29a4-4849-9700-f9f8ebb310d8\") " pod="openshift-dns/node-resolver-hbfpg" Aug 04 11:33:10.267570 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:10.267548 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 11:33:10.267683 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:10.267579 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 11:33:10.267683 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:10.267593 2570 projected.go:196] Error preparing data for projected volume kube-api-access-4w2l4 for pod openshift-network-diagnostics/network-check-target-42v7s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 11:33:10.267683 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:10.267670 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4 podName:f1f75655-126a-410b-ab43-16e96cc67d05 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:10.767652612 +0000 UTC m=+3.012372134 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4w2l4" (UniqueName: "kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4") pod "network-check-target-42v7s" (UID: "f1f75655-126a-410b-ab43-16e96cc67d05") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 11:33:10.268896 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.268852 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xfqt\" (UniqueName: \"kubernetes.io/projected/c1cf79fd-d20b-4d9c-8b7d-e4eefe3e83c6-kube-api-access-8xfqt\") pod \"iptables-alerter-zlp8p\" (UID: \"c1cf79fd-d20b-4d9c-8b7d-e4eefe3e83c6\") " pod="openshift-network-operator/iptables-alerter-zlp8p" Aug 04 11:33:10.268996 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.268954 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc62k\" (UniqueName: \"kubernetes.io/projected/2e8c00dc-adfc-4152-901e-ce48741e273c-kube-api-access-rc62k\") pod \"ovnkube-node-88j8c\" (UID: \"2e8c00dc-adfc-4152-901e-ce48741e273c\") " pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.269166 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.269148 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwjpc\" (UniqueName: \"kubernetes.io/projected/fe775771-24c4-46a8-b53a-4322c76ec540-kube-api-access-zwjpc\") pod \"network-metrics-daemon-kt4m7\" (UID: \"fe775771-24c4-46a8-b53a-4322c76ec540\") " pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:10.269435 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.269414 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wq2c\" (UniqueName: \"kubernetes.io/projected/8f2acb16-9405-4d91-853d-6a54e7a03a59-kube-api-access-4wq2c\") pod \"node-ca-5z7cf\" (UID: \"8f2acb16-9405-4d91-853d-6a54e7a03a59\") " pod="openshift-image-registry/node-ca-5z7cf" Aug 04 11:33:10.270222 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.270201 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57fbz\" (UniqueName: \"kubernetes.io/projected/ad3565b1-c324-4d62-88d6-274e04aaf220-kube-api-access-57fbz\") pod \"multus-additional-cni-plugins-t6mz2\" (UID: \"ad3565b1-c324-4d62-88d6-274e04aaf220\") " pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.356832 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.356750 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" Aug 04 11:33:10.371297 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.371271 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" Aug 04 11:33:10.379878 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.379855 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n4x7h" Aug 04 11:33:10.386554 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.386532 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4wnrp" Aug 04 11:33:10.395132 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.395111 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-t6mz2" Aug 04 11:33:10.403850 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.403825 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hbfpg" Aug 04 11:33:10.409469 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.409435 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5z7cf" Aug 04 11:33:10.418087 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.418067 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zlp8p" Aug 04 11:33:10.422671 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.422651 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:10.661632 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.661556 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret\") pod \"global-pull-secret-syncer-lcqpb\" (UID: \"e3158842-36a1-4213-af82-c6db89481715\") " pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:10.661771 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:10.661702 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 11:33:10.661771 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:10.661766 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret podName:e3158842-36a1-4213-af82-c6db89481715 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:11.66174592 +0000 UTC m=+3.906465453 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret") pod "global-pull-secret-syncer-lcqpb" (UID: "e3158842-36a1-4213-af82-c6db89481715") : object "kube-system"/"original-pull-secret" not registered Aug 04 11:33:10.762124 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.762089 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs\") pod \"network-metrics-daemon-kt4m7\" (UID: \"fe775771-24c4-46a8-b53a-4322c76ec540\") " pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:10.762277 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:10.762210 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 11:33:10.762277 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:10.762271 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs podName:fe775771-24c4-46a8-b53a-4322c76ec540 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:11.762252785 +0000 UTC m=+4.006972312 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs") pod "network-metrics-daemon-kt4m7" (UID: "fe775771-24c4-46a8-b53a-4322c76ec540") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 11:33:10.863356 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:10.863328 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w2l4\" (UniqueName: \"kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4\") pod \"network-check-target-42v7s\" (UID: \"f1f75655-126a-410b-ab43-16e96cc67d05\") " pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:10.863548 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:10.863527 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 11:33:10.863615 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:10.863550 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 11:33:10.863615 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:10.863563 2570 projected.go:196] Error preparing data for projected volume kube-api-access-4w2l4 for pod openshift-network-diagnostics/network-check-target-42v7s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 11:33:10.863711 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:10.863628 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4 podName:f1f75655-126a-410b-ab43-16e96cc67d05 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:11.863605744 +0000 UTC m=+4.108325267 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4w2l4" (UniqueName: "kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4") pod "network-check-target-42v7s" (UID: "f1f75655-126a-410b-ab43-16e96cc67d05") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 11:33:11.108874 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:11.108797 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 11:28:10 +0000 UTC" deadline="2028-04-09 13:56:10.600881121 +0000 UTC" Aug 04 11:33:11.108874 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:11.108829 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14738h22m59.492056024s" Aug 04 11:33:11.121439 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:11.121411 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8dbbbc0_29a4_4849_9700_f9f8ebb310d8.slice/crio-dd88bd8570a795c340fa7feef15280e425ede91edef2068668247b9ba365562d WatchSource:0}: Error finding container dd88bd8570a795c340fa7feef15280e425ede91edef2068668247b9ba365562d: Status 404 returned error can't find the container with id dd88bd8570a795c340fa7feef15280e425ede91edef2068668247b9ba365562d Aug 04 11:33:11.122652 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:11.122625 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8a85cde_1d4e_4c4b_926f_fdec33a673a6.slice/crio-2f4f4bddaa05476233dde9a08d58acd1187fb6e0481930e693075a32da370df9 WatchSource:0}: Error finding container 2f4f4bddaa05476233dde9a08d58acd1187fb6e0481930e693075a32da370df9: Status 404 returned error can't find the container with id 2f4f4bddaa05476233dde9a08d58acd1187fb6e0481930e693075a32da370df9 Aug 04 11:33:11.130505 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:11.130485 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dafc945_e2f2_4b29_8b6d_b27765cc5e6e.slice/crio-7b298ca604151ac236240478b8884187fbca72e0cde462e53f4cdcda21d77e0f WatchSource:0}: Error finding container 7b298ca604151ac236240478b8884187fbca72e0cde462e53f4cdcda21d77e0f: Status 404 returned error can't find the container with id 7b298ca604151ac236240478b8884187fbca72e0cde462e53f4cdcda21d77e0f Aug 04 11:33:11.130829 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:11.130796 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf58b5f4_9fac_4635_badb_1f2b39a971e4.slice/crio-3b355c0bc74a1a02cc8894cc2f544cea35d9c9ace31d6dbe7a3f5f063fd80f56 WatchSource:0}: Error finding container 3b355c0bc74a1a02cc8894cc2f544cea35d9c9ace31d6dbe7a3f5f063fd80f56: Status 404 returned error can't find the container with id 3b355c0bc74a1a02cc8894cc2f544cea35d9c9ace31d6dbe7a3f5f063fd80f56 Aug 04 11:33:11.132184 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:11.132161 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1cf79fd_d20b_4d9c_8b7d_e4eefe3e83c6.slice/crio-cb213d979ad72a0b63b10f0e4138723a846ab589051ef58cbb3cf6470ede2864 WatchSource:0}: Error finding container cb213d979ad72a0b63b10f0e4138723a846ab589051ef58cbb3cf6470ede2864: Status 404 returned error can't find the container with id cb213d979ad72a0b63b10f0e4138723a846ab589051ef58cbb3cf6470ede2864 Aug 04 11:33:11.134261 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:11.134107 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5526e5d1_5421_4f6a_9b97_aa45435fece2.slice/crio-be34a047cf1fa720f73913605494cd07eb8f7e13fd044cef71ac87229f2e2586 WatchSource:0}: Error finding container be34a047cf1fa720f73913605494cd07eb8f7e13fd044cef71ac87229f2e2586: Status 404 returned error can't find the container with id be34a047cf1fa720f73913605494cd07eb8f7e13fd044cef71ac87229f2e2586 Aug 04 11:33:11.135004 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:11.134976 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e8c00dc_adfc_4152_901e_ce48741e273c.slice/crio-a10737f24dad782fc6b118d9e9c2c7494e258c319fe2b2ce14d129f8e6ba3174 WatchSource:0}: Error finding container a10737f24dad782fc6b118d9e9c2c7494e258c319fe2b2ce14d129f8e6ba3174: Status 404 returned error can't find the container with id a10737f24dad782fc6b118d9e9c2c7494e258c319fe2b2ce14d129f8e6ba3174 Aug 04 11:33:11.136009 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:11.135987 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad3565b1_c324_4d62_88d6_274e04aaf220.slice/crio-29e50522ad9c2fd33ebcbb76c8ebcbbd4da0f34ffd8ec9da989e0a269cf762d3 WatchSource:0}: Error finding container 29e50522ad9c2fd33ebcbb76c8ebcbbd4da0f34ffd8ec9da989e0a269cf762d3: Status 404 returned error can't find the container with id 29e50522ad9c2fd33ebcbb76c8ebcbbd4da0f34ffd8ec9da989e0a269cf762d3 Aug 04 11:33:11.188990 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:11.188965 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:11.189105 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:11.189084 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lcqpb" podUID="e3158842-36a1-4213-af82-c6db89481715" Aug 04 11:33:11.197375 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:11.197351 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n4x7h" event={"ID":"b8a85cde-1d4e-4c4b-926f-fdec33a673a6","Type":"ContainerStarted","Data":"2f4f4bddaa05476233dde9a08d58acd1187fb6e0481930e693075a32da370df9"} Aug 04 11:33:11.198426 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:11.198407 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" event={"ID":"2e8c00dc-adfc-4152-901e-ce48741e273c","Type":"ContainerStarted","Data":"a10737f24dad782fc6b118d9e9c2c7494e258c319fe2b2ce14d129f8e6ba3174"} Aug 04 11:33:11.199497 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:11.199473 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" event={"ID":"5526e5d1-5421-4f6a-9b97-aa45435fece2","Type":"ContainerStarted","Data":"be34a047cf1fa720f73913605494cd07eb8f7e13fd044cef71ac87229f2e2586"} Aug 04 11:33:11.200769 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:11.200743 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zlp8p" event={"ID":"c1cf79fd-d20b-4d9c-8b7d-e4eefe3e83c6","Type":"ContainerStarted","Data":"cb213d979ad72a0b63b10f0e4138723a846ab589051ef58cbb3cf6470ede2864"} Aug 04 11:33:11.202055 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:11.201855 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5z7cf" event={"ID":"8f2acb16-9405-4d91-853d-6a54e7a03a59","Type":"ContainerStarted","Data":"fd5c60810b3df57bf5553d7ce7ba3fa1a792be9040780edfade23bb961dabcb2"} Aug 04 11:33:11.203039 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:11.203011 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hbfpg" event={"ID":"f8dbbbc0-29a4-4849-9700-f9f8ebb310d8","Type":"ContainerStarted","Data":"dd88bd8570a795c340fa7feef15280e425ede91edef2068668247b9ba365562d"} Aug 04 11:33:11.204481 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:11.204461 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" event={"ID":"bf58b5f4-9fac-4635-badb-1f2b39a971e4","Type":"ContainerStarted","Data":"3b355c0bc74a1a02cc8894cc2f544cea35d9c9ace31d6dbe7a3f5f063fd80f56"} Aug 04 11:33:11.205697 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:11.205673 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t6mz2" event={"ID":"ad3565b1-c324-4d62-88d6-274e04aaf220","Type":"ContainerStarted","Data":"29e50522ad9c2fd33ebcbb76c8ebcbbd4da0f34ffd8ec9da989e0a269cf762d3"} Aug 04 11:33:11.206858 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:11.206836 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4wnrp" event={"ID":"9dafc945-e2f2-4b29-8b6d-b27765cc5e6e","Type":"ContainerStarted","Data":"7b298ca604151ac236240478b8884187fbca72e0cde462e53f4cdcda21d77e0f"} Aug 04 11:33:11.667791 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:11.667489 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret\") pod \"global-pull-secret-syncer-lcqpb\" (UID: \"e3158842-36a1-4213-af82-c6db89481715\") " pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:11.667791 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:11.667764 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 11:33:11.668001 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:11.667842 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret podName:e3158842-36a1-4213-af82-c6db89481715 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:13.667822688 +0000 UTC m=+5.912542223 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret") pod "global-pull-secret-syncer-lcqpb" (UID: "e3158842-36a1-4213-af82-c6db89481715") : object "kube-system"/"original-pull-secret" not registered Aug 04 11:33:11.768939 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:11.768902 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs\") pod \"network-metrics-daemon-kt4m7\" (UID: \"fe775771-24c4-46a8-b53a-4322c76ec540\") " pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:11.769100 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:11.769081 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 11:33:11.769156 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:11.769147 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs podName:fe775771-24c4-46a8-b53a-4322c76ec540 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:13.769128909 +0000 UTC m=+6.013848431 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs") pod "network-metrics-daemon-kt4m7" (UID: "fe775771-24c4-46a8-b53a-4322c76ec540") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 11:33:11.870053 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:11.870016 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w2l4\" (UniqueName: \"kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4\") pod \"network-check-target-42v7s\" (UID: \"f1f75655-126a-410b-ab43-16e96cc67d05\") " pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:11.870240 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:11.870222 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 11:33:11.870296 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:11.870247 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 11:33:11.870296 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:11.870260 2570 projected.go:196] Error preparing data for projected volume kube-api-access-4w2l4 for pod openshift-network-diagnostics/network-check-target-42v7s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 11:33:11.870397 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:11.870318 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4 podName:f1f75655-126a-410b-ab43-16e96cc67d05 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:13.87030041 +0000 UTC m=+6.115019947 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4w2l4" (UniqueName: "kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4") pod "network-check-target-42v7s" (UID: "f1f75655-126a-410b-ab43-16e96cc67d05") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 11:33:12.109564 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:12.109504 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 11:28:10 +0000 UTC" deadline="2028-02-29 10:30:39.244693886 +0000 UTC" Aug 04 11:33:12.109564 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:12.109562 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13774h57m27.135135869s" Aug 04 11:33:12.190611 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:12.190585 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:12.190767 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:12.190711 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42v7s" podUID="f1f75655-126a-410b-ab43-16e96cc67d05" Aug 04 11:33:12.192586 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:12.192565 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:12.192760 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:12.192680 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4m7" podUID="fe775771-24c4-46a8-b53a-4322c76ec540" Aug 04 11:33:12.223177 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:12.222483 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-47.ec2.internal" event={"ID":"b3c19a22d5a3d4dd837bfebae044979e","Type":"ContainerStarted","Data":"5390dc1b8de567a563fa3293e76196f1e8290724668d7be8d82364ce29410f53"} Aug 04 11:33:12.225934 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:12.225905 2570 generic.go:356] "Generic (PLEG): container finished" podID="138335873249eb34a707d460b9840757" containerID="e444437125f8b757505e09c18fccebd75aa10a1ed5412f0c27819883c0d2a208" exitCode=0 Aug 04 11:33:12.226030 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:12.225953 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal" event={"ID":"138335873249eb34a707d460b9840757","Type":"ContainerDied","Data":"e444437125f8b757505e09c18fccebd75aa10a1ed5412f0c27819883c0d2a208"} Aug 04 11:33:12.254221 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:12.254173 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-47.ec2.internal" podStartSLOduration=3.254153495 podStartE2EDuration="3.254153495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:12.238301757 +0000 UTC m=+4.483021300" watchObservedRunningTime="2026-08-04 11:33:12.254153495 +0000 UTC m=+4.498873038" Aug 04 11:33:13.191193 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:13.191159 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:13.191687 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:13.191299 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lcqpb" podUID="e3158842-36a1-4213-af82-c6db89481715" Aug 04 11:33:13.243151 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:13.243077 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal" event={"ID":"138335873249eb34a707d460b9840757","Type":"ContainerStarted","Data":"00a752c8dbd0af9093d1812f46d08e00be060939a2822bb31afd03527007c0b0"} Aug 04 11:33:13.687747 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:13.687664 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret\") pod \"global-pull-secret-syncer-lcqpb\" (UID: \"e3158842-36a1-4213-af82-c6db89481715\") " pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:13.687913 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:13.687808 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 11:33:13.688348 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:13.687980 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret podName:e3158842-36a1-4213-af82-c6db89481715 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:17.687959497 +0000 UTC m=+9.932679031 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret") pod "global-pull-secret-syncer-lcqpb" (UID: "e3158842-36a1-4213-af82-c6db89481715") : object "kube-system"/"original-pull-secret" not registered Aug 04 11:33:13.788313 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:13.788277 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs\") pod \"network-metrics-daemon-kt4m7\" (UID: \"fe775771-24c4-46a8-b53a-4322c76ec540\") " pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:13.788495 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:13.788430 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 11:33:13.788572 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:13.788507 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs podName:fe775771-24c4-46a8-b53a-4322c76ec540 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:17.78848629 +0000 UTC m=+10.033205825 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs") pod "network-metrics-daemon-kt4m7" (UID: "fe775771-24c4-46a8-b53a-4322c76ec540") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 11:33:13.888931 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:13.888897 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w2l4\" (UniqueName: \"kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4\") pod \"network-check-target-42v7s\" (UID: \"f1f75655-126a-410b-ab43-16e96cc67d05\") " pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:13.889220 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:13.889188 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 11:33:13.889220 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:13.889213 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 11:33:13.889351 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:13.889226 2570 projected.go:196] Error preparing data for projected volume kube-api-access-4w2l4 for pod openshift-network-diagnostics/network-check-target-42v7s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 11:33:13.889351 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:13.889280 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4 podName:f1f75655-126a-410b-ab43-16e96cc67d05 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:17.889262491 +0000 UTC m=+10.133982023 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4w2l4" (UniqueName: "kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4") pod "network-check-target-42v7s" (UID: "f1f75655-126a-410b-ab43-16e96cc67d05") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 11:33:14.189329 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:14.189306 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:14.189504 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:14.189403 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42v7s" podUID="f1f75655-126a-410b-ab43-16e96cc67d05" Aug 04 11:33:14.189711 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:14.189697 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:14.189796 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:14.189777 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4m7" podUID="fe775771-24c4-46a8-b53a-4322c76ec540" Aug 04 11:33:15.189808 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:15.189773 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:15.190248 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:15.189915 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lcqpb" podUID="e3158842-36a1-4213-af82-c6db89481715" Aug 04 11:33:16.193298 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:16.192746 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:16.193298 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:16.192789 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:16.193298 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:16.192884 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4m7" podUID="fe775771-24c4-46a8-b53a-4322c76ec540" Aug 04 11:33:16.193298 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:16.193257 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42v7s" podUID="f1f75655-126a-410b-ab43-16e96cc67d05" Aug 04 11:33:17.189829 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:17.189581 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:17.189829 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:17.189719 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lcqpb" podUID="e3158842-36a1-4213-af82-c6db89481715" Aug 04 11:33:17.724161 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:17.724095 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret\") pod \"global-pull-secret-syncer-lcqpb\" (UID: \"e3158842-36a1-4213-af82-c6db89481715\") " pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:17.724672 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:17.724267 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 11:33:17.724672 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:17.724341 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret podName:e3158842-36a1-4213-af82-c6db89481715 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:25.724321798 +0000 UTC m=+17.969041318 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret") pod "global-pull-secret-syncer-lcqpb" (UID: "e3158842-36a1-4213-af82-c6db89481715") : object "kube-system"/"original-pull-secret" not registered Aug 04 11:33:17.824636 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:17.824590 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs\") pod \"network-metrics-daemon-kt4m7\" (UID: \"fe775771-24c4-46a8-b53a-4322c76ec540\") " pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:17.824829 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:17.824748 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 11:33:17.824829 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:17.824821 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs podName:fe775771-24c4-46a8-b53a-4322c76ec540 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:25.824801226 +0000 UTC m=+18.069520759 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs") pod "network-metrics-daemon-kt4m7" (UID: "fe775771-24c4-46a8-b53a-4322c76ec540") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 11:33:17.925887 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:17.925679 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w2l4\" (UniqueName: \"kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4\") pod \"network-check-target-42v7s\" (UID: \"f1f75655-126a-410b-ab43-16e96cc67d05\") " pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:17.925887 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:17.925841 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 11:33:17.925887 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:17.925860 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 11:33:17.925887 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:17.925872 2570 projected.go:196] Error preparing data for projected volume kube-api-access-4w2l4 for pod openshift-network-diagnostics/network-check-target-42v7s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 11:33:17.926201 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:17.925932 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4 podName:f1f75655-126a-410b-ab43-16e96cc67d05 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:25.925915103 +0000 UTC m=+18.170634626 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4w2l4" (UniqueName: "kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4") pod "network-check-target-42v7s" (UID: "f1f75655-126a-410b-ab43-16e96cc67d05") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 11:33:18.190474 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:18.190365 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:18.190633 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:18.190515 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4m7" podUID="fe775771-24c4-46a8-b53a-4322c76ec540" Aug 04 11:33:18.190928 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:18.190754 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:18.190928 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:18.190886 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42v7s" podUID="f1f75655-126a-410b-ab43-16e96cc67d05" Aug 04 11:33:19.189779 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:19.189749 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:19.190243 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:19.189858 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lcqpb" podUID="e3158842-36a1-4213-af82-c6db89481715" Aug 04 11:33:20.189508 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:20.189359 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:20.189707 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:20.189521 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42v7s" podUID="f1f75655-126a-410b-ab43-16e96cc67d05" Aug 04 11:33:20.189707 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:20.189575 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:20.189707 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:20.189691 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4m7" podUID="fe775771-24c4-46a8-b53a-4322c76ec540" Aug 04 11:33:21.189812 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:21.189778 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:21.190262 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:21.189910 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lcqpb" podUID="e3158842-36a1-4213-af82-c6db89481715" Aug 04 11:33:22.189264 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:22.189225 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:22.189439 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:22.189373 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4m7" podUID="fe775771-24c4-46a8-b53a-4322c76ec540" Aug 04 11:33:22.189439 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:22.189405 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:22.189554 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:22.189530 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42v7s" podUID="f1f75655-126a-410b-ab43-16e96cc67d05" Aug 04 11:33:23.189281 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:23.189251 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:23.189749 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:23.189374 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lcqpb" podUID="e3158842-36a1-4213-af82-c6db89481715" Aug 04 11:33:24.189199 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:24.189163 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:24.189392 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:24.189168 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:24.189392 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:24.189292 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42v7s" podUID="f1f75655-126a-410b-ab43-16e96cc67d05" Aug 04 11:33:24.189392 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:24.189355 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4m7" podUID="fe775771-24c4-46a8-b53a-4322c76ec540" Aug 04 11:33:25.188886 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:25.188851 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:25.189095 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:25.188950 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lcqpb" podUID="e3158842-36a1-4213-af82-c6db89481715" Aug 04 11:33:25.789409 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:25.789377 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret\") pod \"global-pull-secret-syncer-lcqpb\" (UID: \"e3158842-36a1-4213-af82-c6db89481715\") " pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:25.789866 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:25.789557 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 11:33:25.789866 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:25.789633 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret podName:e3158842-36a1-4213-af82-c6db89481715 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:41.789613776 +0000 UTC m=+34.034333295 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret") pod "global-pull-secret-syncer-lcqpb" (UID: "e3158842-36a1-4213-af82-c6db89481715") : object "kube-system"/"original-pull-secret" not registered Aug 04 11:33:25.889779 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:25.889748 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs\") pod \"network-metrics-daemon-kt4m7\" (UID: \"fe775771-24c4-46a8-b53a-4322c76ec540\") " pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:25.889964 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:25.889899 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 11:33:25.890023 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:25.889969 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs podName:fe775771-24c4-46a8-b53a-4322c76ec540 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:41.889948426 +0000 UTC m=+34.134667949 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs") pod "network-metrics-daemon-kt4m7" (UID: "fe775771-24c4-46a8-b53a-4322c76ec540") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 11:33:25.990338 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:25.990295 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w2l4\" (UniqueName: \"kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4\") pod \"network-check-target-42v7s\" (UID: \"f1f75655-126a-410b-ab43-16e96cc67d05\") " pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:25.990525 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:25.990507 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 11:33:25.990611 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:25.990529 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 11:33:25.990611 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:25.990541 2570 projected.go:196] Error preparing data for projected volume kube-api-access-4w2l4 for pod openshift-network-diagnostics/network-check-target-42v7s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 11:33:25.990611 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:25.990604 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4 podName:f1f75655-126a-410b-ab43-16e96cc67d05 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:41.990585124 +0000 UTC m=+34.235304658 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4w2l4" (UniqueName: "kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4") pod "network-check-target-42v7s" (UID: "f1f75655-126a-410b-ab43-16e96cc67d05") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 11:33:26.192053 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:26.191973 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:26.192053 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:26.192013 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:26.192246 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:26.192134 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4m7" podUID="fe775771-24c4-46a8-b53a-4322c76ec540" Aug 04 11:33:26.192305 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:26.192269 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42v7s" podUID="f1f75655-126a-410b-ab43-16e96cc67d05" Aug 04 11:33:27.189047 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:27.189010 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:27.189513 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:27.189121 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lcqpb" podUID="e3158842-36a1-4213-af82-c6db89481715" Aug 04 11:33:28.192586 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:28.192553 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:28.193034 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:28.192693 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4m7" podUID="fe775771-24c4-46a8-b53a-4322c76ec540" Aug 04 11:33:28.193034 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:28.192749 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:28.193034 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:28.192865 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42v7s" podUID="f1f75655-126a-410b-ab43-16e96cc67d05" Aug 04 11:33:29.189541 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:29.189506 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:29.189679 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:29.189612 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lcqpb" podUID="e3158842-36a1-4213-af82-c6db89481715" Aug 04 11:33:30.189918 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:30.189060 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:30.189918 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:30.189070 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:30.189918 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:30.189310 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42v7s" podUID="f1f75655-126a-410b-ab43-16e96cc67d05" Aug 04 11:33:30.189918 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:30.189395 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4m7" podUID="fe775771-24c4-46a8-b53a-4322c76ec540" Aug 04 11:33:30.280172 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:30.280024 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5z7cf" event={"ID":"8f2acb16-9405-4d91-853d-6a54e7a03a59","Type":"ContainerStarted","Data":"14645cb051c4d6ff9789a13189e2690e3bea7298955bede6af1dc421a3c184fb"} Aug 04 11:33:30.281458 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:30.281423 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hbfpg" event={"ID":"f8dbbbc0-29a4-4849-9700-f9f8ebb310d8","Type":"ContainerStarted","Data":"f36db8ea190fcfd81399c0628cc91a6a25e9c3d30d788570382970b4020c569f"} Aug 04 11:33:30.282773 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:30.282750 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" event={"ID":"bf58b5f4-9fac-4635-badb-1f2b39a971e4","Type":"ContainerStarted","Data":"24f58ad9e8964eea8a47b4e083b2426691b7c4026035c9a993f02e200c5bed69"} Aug 04 11:33:30.284024 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:30.283994 2570 generic.go:356] "Generic (PLEG): container finished" podID="ad3565b1-c324-4d62-88d6-274e04aaf220" containerID="b4ace96ea9ee1c3a14e93261b9bd4391b1a465d7cd45063b81da794f4d2af891" exitCode=0 Aug 04 11:33:30.284119 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:30.284069 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t6mz2" event={"ID":"ad3565b1-c324-4d62-88d6-274e04aaf220","Type":"ContainerDied","Data":"b4ace96ea9ee1c3a14e93261b9bd4391b1a465d7cd45063b81da794f4d2af891"} Aug 04 11:33:30.285353 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:30.285336 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4wnrp" event={"ID":"9dafc945-e2f2-4b29-8b6d-b27765cc5e6e","Type":"ContainerStarted","Data":"b95739df8ab386fa4f9dba29ecb3f3f34077daa24299103a403ba08222de1a14"} Aug 04 11:33:30.286717 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:30.286684 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n4x7h" event={"ID":"b8a85cde-1d4e-4c4b-926f-fdec33a673a6","Type":"ContainerStarted","Data":"9bb879fad2f33d8cf8365571659afa25e84e6cd873f452c031043685a9a57abe"} Aug 04 11:33:30.288427 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:30.288408 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 11:33:30.288779 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:30.288759 2570 generic.go:356] "Generic (PLEG): container finished" podID="2e8c00dc-adfc-4152-901e-ce48741e273c" containerID="e9bd954ff059267c74edd3142c9713a86594ceb7d1514f623e382d8de34e0220" exitCode=1 Aug 04 11:33:30.288846 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:30.288827 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" event={"ID":"2e8c00dc-adfc-4152-901e-ce48741e273c","Type":"ContainerStarted","Data":"e525add762222f555d8d4007b49eed24340f2bb5df44193cf2a3a3a50462a257"} Aug 04 11:33:30.288918 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:30.288899 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" event={"ID":"2e8c00dc-adfc-4152-901e-ce48741e273c","Type":"ContainerDied","Data":"e9bd954ff059267c74edd3142c9713a86594ceb7d1514f623e382d8de34e0220"} Aug 04 11:33:30.289014 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:30.288956 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" event={"ID":"2e8c00dc-adfc-4152-901e-ce48741e273c","Type":"ContainerStarted","Data":"f0bf95fcdef6ca87403db52e0ae90f97ee9963bffb412d914513cf91d353e0ad"} Aug 04 11:33:30.290169 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:30.290153 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" event={"ID":"5526e5d1-5421-4f6a-9b97-aa45435fece2","Type":"ContainerStarted","Data":"64b76afd5eb51133ddf51e3a28da832db075676658324a8c915473473e62fa30"} Aug 04 11:33:30.297085 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:30.297052 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5z7cf" podStartSLOduration=3.643862085 podStartE2EDuration="22.297041564s" totalImagesPullingTime="18.653179479s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:30.296436725 +0000 UTC m=+22.541156268" watchObservedRunningTime="2026-08-04 11:33:30.297041564 +0000 UTC m=+22.541761106" Aug 04 11:33:30.297179 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:30.297114 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal" podStartSLOduration=21.297109466 podStartE2EDuration="21.297109466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:13.268016382 +0000 UTC m=+5.512735925" watchObservedRunningTime="2026-08-04 11:33:30.297109466 +0000 UTC m=+22.541829008" Aug 04 11:33:30.314562 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:30.314527 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rjrxh" podStartSLOduration=3.689504443 podStartE2EDuration="22.314516569s" totalImagesPullingTime="18.625012126s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:30.31433609 +0000 UTC m=+22.559055633" watchObservedRunningTime="2026-08-04 11:33:30.314516569 +0000 UTC m=+22.559236110" Aug 04 11:33:30.335037 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:30.334992 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-n4x7h" podStartSLOduration=3.678285668 podStartE2EDuration="22.334980674s" totalImagesPullingTime="18.656695006s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:30.334315312 +0000 UTC m=+22.579034853" watchObservedRunningTime="2026-08-04 11:33:30.334980674 +0000 UTC m=+22.579700216" Aug 04 11:33:30.351098 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:30.351063 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4wnrp" podStartSLOduration=8.055953644 podStartE2EDuration="22.35105166s" totalImagesPullingTime="14.295098016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:33:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:30.350795332 +0000 UTC m=+22.595514874" watchObservedRunningTime="2026-08-04 11:33:30.35105166 +0000 UTC m=+22.595771202" Aug 04 11:33:30.356874 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:30.356860 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4wnrp" Aug 04 11:33:30.400984 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:30.400940 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hbfpg" podStartSLOduration=3.747070448 podStartE2EDuration="22.400926673s" totalImagesPullingTime="18.653856225s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:30.368089991 +0000 UTC m=+22.612809529" watchObservedRunningTime="2026-08-04 11:33:30.400926673 +0000 UTC m=+22.645646212" Aug 04 11:33:31.189475 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:31.189397 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:31.189635 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:31.189511 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lcqpb" podUID="e3158842-36a1-4213-af82-c6db89481715" Aug 04 11:33:31.295744 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:31.295708 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 11:33:31.296531 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:31.296493 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" event={"ID":"2e8c00dc-adfc-4152-901e-ce48741e273c","Type":"ContainerStarted","Data":"0476ac13ba8f427f0d1761315112b3a15db788927a4839acb0ecc4b2f5024f77"} Aug 04 11:33:31.296659 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:31.296578 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" event={"ID":"2e8c00dc-adfc-4152-901e-ce48741e273c","Type":"ContainerStarted","Data":"3d0edf9f9e36f49316fbe651d6996e9544c6ce9e3fb8c7ce2585ddbebe5a2c44"} Aug 04 11:33:31.296659 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:31.296627 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" event={"ID":"2e8c00dc-adfc-4152-901e-ce48741e273c","Type":"ContainerStarted","Data":"1c6f778871ca699fdcdc56a8f2145ca6d134f4f1ec5a7ef8c42f5db555498332"} Aug 04 11:33:31.298218 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:31.298193 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zlp8p" event={"ID":"c1cf79fd-d20b-4d9c-8b7d-e4eefe3e83c6","Type":"ContainerStarted","Data":"7d23915c139e13f1d46739fe8e56e34c03140424afb46430c446f3affa4e7f0c"} Aug 04 11:33:31.320166 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:31.320119 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zlp8p" podStartSLOduration=4.672914434 podStartE2EDuration="23.320100216s" totalImagesPullingTime="18.647185782s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:31.319646438 +0000 UTC m=+23.564365981" watchObservedRunningTime="2026-08-04 11:33:31.320100216 +0000 UTC m=+23.564819789" Aug 04 11:33:31.488497 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:31.488469 2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Aug 04 11:33:32.106687 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:32.106573 2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-04T11:33:31.488486624Z","UUID":"727c08c6-773c-4b11-93c8-8bf4f249901f","Handler":null,"Name":"","Endpoint":""} Aug 04 11:33:32.108170 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:32.108154 2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Aug 04 11:33:32.108281 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:32.108190 2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Aug 04 11:33:32.189870 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:32.189837 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:32.189870 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:32.189860 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:32.190087 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:32.189967 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4m7" podUID="fe775771-24c4-46a8-b53a-4322c76ec540" Aug 04 11:33:32.190087 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:32.190071 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42v7s" podUID="f1f75655-126a-410b-ab43-16e96cc67d05" Aug 04 11:33:32.302609 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:32.302573 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" event={"ID":"bf58b5f4-9fac-4635-badb-1f2b39a971e4","Type":"ContainerStarted","Data":"90650c7a161d928cb5a1ae08cda56a0900f8667fbbe7446dd146383d7720fdd2"} Aug 04 11:33:33.189599 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:33.189569 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:33.189776 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:33.189690 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lcqpb" podUID="e3158842-36a1-4213-af82-c6db89481715" Aug 04 11:33:34.190056 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:34.189815 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:34.190523 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:34.189877 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:34.190523 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:34.190177 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4m7" podUID="fe775771-24c4-46a8-b53a-4322c76ec540" Aug 04 11:33:34.190523 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:34.190236 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42v7s" podUID="f1f75655-126a-410b-ab43-16e96cc67d05" Aug 04 11:33:35.188812 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:35.188781 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:35.189041 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:35.188879 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lcqpb" podUID="e3158842-36a1-4213-af82-c6db89481715" Aug 04 11:33:35.265662 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:35.265637 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4wnrp" Aug 04 11:33:35.266241 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:35.266167 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4wnrp" Aug 04 11:33:35.310678 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:35.310653 2570 generic.go:356] "Generic (PLEG): container finished" podID="ad3565b1-c324-4d62-88d6-274e04aaf220" containerID="20f9f2504ebdf9ebb97c1261cf7a094e5adbb2d39cc8322297f7140cb4571ae8" exitCode=0 Aug 04 11:33:35.310828 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:35.310710 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t6mz2" event={"ID":"ad3565b1-c324-4d62-88d6-274e04aaf220","Type":"ContainerDied","Data":"20f9f2504ebdf9ebb97c1261cf7a094e5adbb2d39cc8322297f7140cb4571ae8"} Aug 04 11:33:35.313820 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:35.313805 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 11:33:35.314220 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:35.314196 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" event={"ID":"2e8c00dc-adfc-4152-901e-ce48741e273c","Type":"ContainerStarted","Data":"b802a5ca7820f29ff6a31a7b08f578dd17c6539f93d4c7acf04330ab241d92fd"} Aug 04 11:33:35.316034 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:35.316002 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" event={"ID":"bf58b5f4-9fac-4635-badb-1f2b39a971e4","Type":"ContainerStarted","Data":"f309ff9a6cb4a5395dfb1beb1d74b483a2dd16d75e7ee999ff8ac6848473c7a5"} Aug 04 11:33:35.316659 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:35.316644 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4wnrp" Aug 04 11:33:35.358008 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:35.357961 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kdkpv" podStartSLOduration=4.690394018 podStartE2EDuration="27.357948404s" totalImagesPullingTime="22.667554386s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:35.357776674 +0000 UTC m=+27.602496217" watchObservedRunningTime="2026-08-04 11:33:35.357948404 +0000 UTC m=+27.602667974" Aug 04 11:33:36.189782 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:36.189754 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:36.189940 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:36.189860 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4m7" podUID="fe775771-24c4-46a8-b53a-4322c76ec540" Aug 04 11:33:36.189940 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:36.189912 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:36.190020 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:36.189986 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42v7s" podUID="f1f75655-126a-410b-ab43-16e96cc67d05" Aug 04 11:33:37.189303 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:37.189126 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:37.189859 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:37.189367 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lcqpb" podUID="e3158842-36a1-4213-af82-c6db89481715" Aug 04 11:33:37.323277 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:37.323252 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 11:33:37.323626 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:37.323605 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" event={"ID":"2e8c00dc-adfc-4152-901e-ce48741e273c","Type":"ContainerStarted","Data":"0e63f0cf5ec0bde0b4ab41a2fa0bcf1c526fd6bf8bcf9b7a3e43e0ea5c71d074"} Aug 04 11:33:37.323925 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:37.323892 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:37.324048 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:37.324031 2570 scope.go:117] "RemoveContainer" containerID="e9bd954ff059267c74edd3142c9713a86594ceb7d1514f623e382d8de34e0220" Aug 04 11:33:37.325801 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:37.325782 2570 generic.go:356] "Generic (PLEG): container finished" podID="ad3565b1-c324-4d62-88d6-274e04aaf220" containerID="d025c4bdd20de8a555874c4c66002b60b8a1d0937d2ae8c5c1950a6cc5671c2d" exitCode=0 Aug 04 11:33:37.325801 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:37.325810 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t6mz2" event={"ID":"ad3565b1-c324-4d62-88d6-274e04aaf220","Type":"ContainerDied","Data":"d025c4bdd20de8a555874c4c66002b60b8a1d0937d2ae8c5c1950a6cc5671c2d"} Aug 04 11:33:37.340003 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:37.339988 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:38.190829 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:38.190801 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:38.191263 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:38.190915 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4m7" podUID="fe775771-24c4-46a8-b53a-4322c76ec540" Aug 04 11:33:38.191263 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:38.190925 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:38.191263 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:38.190997 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42v7s" podUID="f1f75655-126a-410b-ab43-16e96cc67d05" Aug 04 11:33:38.333867 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:38.333840 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 11:33:38.334283 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:38.334248 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" event={"ID":"2e8c00dc-adfc-4152-901e-ce48741e273c","Type":"ContainerStarted","Data":"b0fe663eb83bd7773b1a87e9554016c0fa9ba29343ac3e8fd98976adecfdffaf"} Aug 04 11:33:38.334520 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:38.334504 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:38.334593 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:38.334576 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:38.352398 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:38.352381 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:33:38.362989 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:38.362948 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" podStartSLOduration=11.516804876 podStartE2EDuration="30.362935002s" totalImagesPullingTime="18.846130126s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:38.36115921 +0000 UTC m=+30.605878751" watchObservedRunningTime="2026-08-04 11:33:38.362935002 +0000 UTC m=+30.607654544" Aug 04 11:33:38.871510 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:38.871248 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-42v7s"] Aug 04 11:33:38.871667 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:38.871597 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:38.871727 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:38.871699 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42v7s" podUID="f1f75655-126a-410b-ab43-16e96cc67d05" Aug 04 11:33:38.871815 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:38.871796 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lcqpb"] Aug 04 11:33:38.871925 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:38.871878 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:38.871982 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:38.871954 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lcqpb" podUID="e3158842-36a1-4213-af82-c6db89481715" Aug 04 11:33:38.874022 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:38.873879 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kt4m7"] Aug 04 11:33:38.874022 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:38.873979 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:38.874181 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:38.874088 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4m7" podUID="fe775771-24c4-46a8-b53a-4322c76ec540" Aug 04 11:33:39.339617 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:39.339582 2570 generic.go:356] "Generic (PLEG): container finished" podID="ad3565b1-c324-4d62-88d6-274e04aaf220" containerID="670264c110d9b44e576717076aca4de43828b3da4831fae81ba1c47599f742b5" exitCode=0 Aug 04 11:33:39.340053 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:39.339665 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t6mz2" event={"ID":"ad3565b1-c324-4d62-88d6-274e04aaf220","Type":"ContainerDied","Data":"670264c110d9b44e576717076aca4de43828b3da4831fae81ba1c47599f742b5"} Aug 04 11:33:41.189560 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:41.189528 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:41.190204 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:41.189536 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:41.190204 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:41.189651 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42v7s" podUID="f1f75655-126a-410b-ab43-16e96cc67d05" Aug 04 11:33:41.190204 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:41.189537 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:41.190204 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:41.189762 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4m7" podUID="fe775771-24c4-46a8-b53a-4322c76ec540" Aug 04 11:33:41.190204 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:41.189872 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lcqpb" podUID="e3158842-36a1-4213-af82-c6db89481715" Aug 04 11:33:41.805122 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:41.805087 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret\") pod \"global-pull-secret-syncer-lcqpb\" (UID: \"e3158842-36a1-4213-af82-c6db89481715\") " pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:41.805291 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:41.805241 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 11:33:41.805340 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:41.805300 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret podName:e3158842-36a1-4213-af82-c6db89481715 nodeName:}" failed. No retries permitted until 2026-08-04 11:34:13.805284119 +0000 UTC m=+66.050003653 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret") pod "global-pull-secret-syncer-lcqpb" (UID: "e3158842-36a1-4213-af82-c6db89481715") : object "kube-system"/"original-pull-secret" not registered Aug 04 11:33:41.906002 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:41.905964 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs\") pod \"network-metrics-daemon-kt4m7\" (UID: \"fe775771-24c4-46a8-b53a-4322c76ec540\") " pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:41.906169 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:41.906138 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 11:33:41.906228 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:41.906218 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs podName:fe775771-24c4-46a8-b53a-4322c76ec540 nodeName:}" failed. No retries permitted until 2026-08-04 11:34:13.90619689 +0000 UTC m=+66.150916415 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs") pod "network-metrics-daemon-kt4m7" (UID: "fe775771-24c4-46a8-b53a-4322c76ec540") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 11:33:42.006484 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:42.006430 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w2l4\" (UniqueName: \"kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4\") pod \"network-check-target-42v7s\" (UID: \"f1f75655-126a-410b-ab43-16e96cc67d05\") " pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:42.006645 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:42.006603 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 11:33:42.006645 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:42.006625 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 11:33:42.006645 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:42.006635 2570 projected.go:196] Error preparing data for projected volume kube-api-access-4w2l4 for pod openshift-network-diagnostics/network-check-target-42v7s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 11:33:42.006770 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:42.006695 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4 podName:f1f75655-126a-410b-ab43-16e96cc67d05 nodeName:}" failed. No retries permitted until 2026-08-04 11:34:14.00667722 +0000 UTC m=+66.251396739 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4w2l4" (UniqueName: "kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4") pod "network-check-target-42v7s" (UID: "f1f75655-126a-410b-ab43-16e96cc67d05") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 11:33:43.189205 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.189173 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:43.189672 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.189217 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:43.189672 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.189185 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:43.189672 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:43.189313 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kt4m7" podUID="fe775771-24c4-46a8-b53a-4322c76ec540" Aug 04 11:33:43.189672 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:43.189381 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42v7s" podUID="f1f75655-126a-410b-ab43-16e96cc67d05" Aug 04 11:33:43.189672 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:43.189463 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lcqpb" podUID="e3158842-36a1-4213-af82-c6db89481715" Aug 04 11:33:43.664545 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.664518 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeReady" Aug 04 11:33:43.664646 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.664640 2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Aug 04 11:33:43.699900 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.699861 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-675cff8469-qlt5l"] Aug 04 11:33:43.702603 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.702588 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:43.706354 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.706337 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Aug 04 11:33:43.706440 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.706398 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gql25\"" Aug 04 11:33:43.706502 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.706488 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Aug 04 11:33:43.706772 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.706758 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Aug 04 11:33:43.711427 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.711405 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Aug 04 11:33:43.712235 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.712217 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pjxbb"] Aug 04 11:33:43.715762 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.715720 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pjxbb" Aug 04 11:33:43.716134 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.716119 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-db4rq"] Aug 04 11:33:43.718057 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.718040 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Aug 04 11:33:43.718134 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.718089 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Aug 04 11:33:43.718177 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.718137 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hll8p\"" Aug 04 11:33:43.719311 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.719122 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-db4rq" Aug 04 11:33:43.720186 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.720167 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-675cff8469-qlt5l"] Aug 04 11:33:43.721682 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.721664 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Aug 04 11:33:43.721806 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.721789 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tcjzq\"" Aug 04 11:33:43.721864 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.721829 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Aug 04 11:33:43.721927 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.721870 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Aug 04 11:33:43.731800 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.731782 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pjxbb"] Aug 04 11:33:43.732473 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.732436 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-db4rq"] Aug 04 11:33:43.820136 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.820105 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-image-registry-private-configuration\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:43.820285 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.820144 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84257250-96bb-4afa-91fa-f2f031eaa074-metrics-tls\") pod \"dns-default-pjxbb\" (UID: \"84257250-96bb-4afa-91fa-f2f031eaa074\") " pod="openshift-dns/dns-default-pjxbb" Aug 04 11:33:43.820285 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.820164 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-installation-pull-secrets\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:43.820285 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.820197 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-trusted-ca\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:43.820285 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.820214 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsskd\" (UniqueName: \"kubernetes.io/projected/84257250-96bb-4afa-91fa-f2f031eaa074-kube-api-access-bsskd\") pod \"dns-default-pjxbb\" (UID: \"84257250-96bb-4afa-91fa-f2f031eaa074\") " pod="openshift-dns/dns-default-pjxbb" Aug 04 11:33:43.820285 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.820229 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlcpr\" (UniqueName: \"kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-kube-api-access-jlcpr\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:43.820285 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.820252 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-ca-trust-extracted\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:43.820524 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.820303 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-certificates\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:43.820524 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.820327 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-bound-sa-token\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:43.820524 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.820368 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e7bd05e-d544-4ac5-b099-612ea158fc36-cert\") pod \"ingress-canary-db4rq\" (UID: \"7e7bd05e-d544-4ac5-b099-612ea158fc36\") " pod="openshift-ingress-canary/ingress-canary-db4rq" Aug 04 11:33:43.820524 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.820391 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/84257250-96bb-4afa-91fa-f2f031eaa074-config-volume\") pod \"dns-default-pjxbb\" (UID: \"84257250-96bb-4afa-91fa-f2f031eaa074\") " pod="openshift-dns/dns-default-pjxbb" Aug 04 11:33:43.820524 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.820413 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84257250-96bb-4afa-91fa-f2f031eaa074-tmp-dir\") pod \"dns-default-pjxbb\" (UID: \"84257250-96bb-4afa-91fa-f2f031eaa074\") " pod="openshift-dns/dns-default-pjxbb" Aug 04 11:33:43.820524 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.820434 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jbzv\" (UniqueName: \"kubernetes.io/projected/7e7bd05e-d544-4ac5-b099-612ea158fc36-kube-api-access-9jbzv\") pod \"ingress-canary-db4rq\" (UID: \"7e7bd05e-d544-4ac5-b099-612ea158fc36\") " pod="openshift-ingress-canary/ingress-canary-db4rq" Aug 04 11:33:43.820524 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.820468 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:43.921431 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.921387 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-trusted-ca\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:43.921431 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.921435 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsskd\" (UniqueName: \"kubernetes.io/projected/84257250-96bb-4afa-91fa-f2f031eaa074-kube-api-access-bsskd\") pod \"dns-default-pjxbb\" (UID: \"84257250-96bb-4afa-91fa-f2f031eaa074\") " pod="openshift-dns/dns-default-pjxbb" Aug 04 11:33:43.921581 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.921470 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlcpr\" (UniqueName: \"kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-kube-api-access-jlcpr\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:43.921619 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.921601 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-ca-trust-extracted\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:43.921656 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.921639 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-certificates\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:43.921695 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.921658 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-bound-sa-token\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:43.921695 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.921674 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e7bd05e-d544-4ac5-b099-612ea158fc36-cert\") pod \"ingress-canary-db4rq\" (UID: \"7e7bd05e-d544-4ac5-b099-612ea158fc36\") " pod="openshift-ingress-canary/ingress-canary-db4rq" Aug 04 11:33:43.921783 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.921710 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/84257250-96bb-4afa-91fa-f2f031eaa074-config-volume\") pod \"dns-default-pjxbb\" (UID: \"84257250-96bb-4afa-91fa-f2f031eaa074\") " pod="openshift-dns/dns-default-pjxbb" Aug 04 11:33:43.921783 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.921733 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84257250-96bb-4afa-91fa-f2f031eaa074-tmp-dir\") pod \"dns-default-pjxbb\" (UID: \"84257250-96bb-4afa-91fa-f2f031eaa074\") " pod="openshift-dns/dns-default-pjxbb" Aug 04 11:33:43.921783 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.921754 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jbzv\" (UniqueName: \"kubernetes.io/projected/7e7bd05e-d544-4ac5-b099-612ea158fc36-kube-api-access-9jbzv\") pod \"ingress-canary-db4rq\" (UID: \"7e7bd05e-d544-4ac5-b099-612ea158fc36\") " pod="openshift-ingress-canary/ingress-canary-db4rq" Aug 04 11:33:43.921783 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.921781 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:43.922182 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.921834 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-image-registry-private-configuration\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:43.922182 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.921884 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84257250-96bb-4afa-91fa-f2f031eaa074-metrics-tls\") pod \"dns-default-pjxbb\" (UID: \"84257250-96bb-4afa-91fa-f2f031eaa074\") " pod="openshift-dns/dns-default-pjxbb" Aug 04 11:33:43.922182 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.921909 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-installation-pull-secrets\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:43.922182 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:43.922070 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 11:33:43.922182 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.922073 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84257250-96bb-4afa-91fa-f2f031eaa074-tmp-dir\") pod \"dns-default-pjxbb\" (UID: \"84257250-96bb-4afa-91fa-f2f031eaa074\") " pod="openshift-dns/dns-default-pjxbb" Aug 04 11:33:43.922182 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:43.922086 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-675cff8469-qlt5l: secret "image-registry-tls" not found Aug 04 11:33:43.922182 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:43.922166 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls podName:2b083227-6cfe-42a3-a2a6-14f733c5e5d2 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:44.422146478 +0000 UTC m=+36.666866018 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls") pod "image-registry-675cff8469-qlt5l" (UID: "2b083227-6cfe-42a3-a2a6-14f733c5e5d2") : secret "image-registry-tls" not found Aug 04 11:33:43.922545 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.922216 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-certificates\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:43.922545 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:43.922397 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 11:33:43.922545 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.922403 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/84257250-96bb-4afa-91fa-f2f031eaa074-config-volume\") pod \"dns-default-pjxbb\" (UID: \"84257250-96bb-4afa-91fa-f2f031eaa074\") " pod="openshift-dns/dns-default-pjxbb" Aug 04 11:33:43.922545 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.922421 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-ca-trust-extracted\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:43.922545 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.922395 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-trusted-ca\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:43.922545 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:43.922480 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84257250-96bb-4afa-91fa-f2f031eaa074-metrics-tls podName:84257250-96bb-4afa-91fa-f2f031eaa074 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:44.42246421 +0000 UTC m=+36.667183745 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/84257250-96bb-4afa-91fa-f2f031eaa074-metrics-tls") pod "dns-default-pjxbb" (UID: "84257250-96bb-4afa-91fa-f2f031eaa074") : secret "dns-default-metrics-tls" not found Aug 04 11:33:43.925832 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.925814 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-installation-pull-secrets\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:43.925832 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.925828 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e7bd05e-d544-4ac5-b099-612ea158fc36-cert\") pod \"ingress-canary-db4rq\" (UID: \"7e7bd05e-d544-4ac5-b099-612ea158fc36\") " pod="openshift-ingress-canary/ingress-canary-db4rq" Aug 04 11:33:43.925832 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.925819 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-image-registry-private-configuration\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:43.932834 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.932806 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsskd\" (UniqueName: \"kubernetes.io/projected/84257250-96bb-4afa-91fa-f2f031eaa074-kube-api-access-bsskd\") pod \"dns-default-pjxbb\" (UID: \"84257250-96bb-4afa-91fa-f2f031eaa074\") " pod="openshift-dns/dns-default-pjxbb" Aug 04 11:33:43.932927 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.932836 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-bound-sa-token\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:43.932970 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.932947 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jbzv\" (UniqueName: \"kubernetes.io/projected/7e7bd05e-d544-4ac5-b099-612ea158fc36-kube-api-access-9jbzv\") pod \"ingress-canary-db4rq\" (UID: \"7e7bd05e-d544-4ac5-b099-612ea158fc36\") " pod="openshift-ingress-canary/ingress-canary-db4rq" Aug 04 11:33:43.933058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:43.933038 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlcpr\" (UniqueName: \"kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-kube-api-access-jlcpr\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:44.029868 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:44.029839 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-db4rq" Aug 04 11:33:44.159074 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:44.158906 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-db4rq"] Aug 04 11:33:44.162040 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:33:44.162013 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e7bd05e_d544_4ac5_b099_612ea158fc36.slice/crio-6ca7bc86ee3c84cb3391b6db2ebb7c8f6bf39565120481a8e11ee6c6ca893cf0 WatchSource:0}: Error finding container 6ca7bc86ee3c84cb3391b6db2ebb7c8f6bf39565120481a8e11ee6c6ca893cf0: Status 404 returned error can't find the container with id 6ca7bc86ee3c84cb3391b6db2ebb7c8f6bf39565120481a8e11ee6c6ca893cf0 Aug 04 11:33:44.353241 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:44.353125 2570 generic.go:356] "Generic (PLEG): container finished" podID="ad3565b1-c324-4d62-88d6-274e04aaf220" containerID="de4b13a2370108d0a0e1d8a328c9a1d46ea5070b85316ff5bed969b1a3a8bec7" exitCode=0 Aug 04 11:33:44.353241 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:44.353198 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t6mz2" event={"ID":"ad3565b1-c324-4d62-88d6-274e04aaf220","Type":"ContainerDied","Data":"de4b13a2370108d0a0e1d8a328c9a1d46ea5070b85316ff5bed969b1a3a8bec7"} Aug 04 11:33:44.354629 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:44.354411 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-db4rq" event={"ID":"7e7bd05e-d544-4ac5-b099-612ea158fc36","Type":"ContainerStarted","Data":"6ca7bc86ee3c84cb3391b6db2ebb7c8f6bf39565120481a8e11ee6c6ca893cf0"} Aug 04 11:33:44.425541 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:44.425516 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:44.425644 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:44.425605 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84257250-96bb-4afa-91fa-f2f031eaa074-metrics-tls\") pod \"dns-default-pjxbb\" (UID: \"84257250-96bb-4afa-91fa-f2f031eaa074\") " pod="openshift-dns/dns-default-pjxbb" Aug 04 11:33:44.425696 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:44.425673 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 11:33:44.425696 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:44.425689 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-675cff8469-qlt5l: secret "image-registry-tls" not found Aug 04 11:33:44.425764 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:44.425743 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls podName:2b083227-6cfe-42a3-a2a6-14f733c5e5d2 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:45.425726052 +0000 UTC m=+37.670445575 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls") pod "image-registry-675cff8469-qlt5l" (UID: "2b083227-6cfe-42a3-a2a6-14f733c5e5d2") : secret "image-registry-tls" not found Aug 04 11:33:44.425810 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:44.425767 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 11:33:44.425847 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:44.425830 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84257250-96bb-4afa-91fa-f2f031eaa074-metrics-tls podName:84257250-96bb-4afa-91fa-f2f031eaa074 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:45.425812211 +0000 UTC m=+37.670531752 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/84257250-96bb-4afa-91fa-f2f031eaa074-metrics-tls") pod "dns-default-pjxbb" (UID: "84257250-96bb-4afa-91fa-f2f031eaa074") : secret "dns-default-metrics-tls" not found Aug 04 11:33:45.189229 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:45.189168 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:33:45.189229 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:45.189195 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:33:45.189229 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:45.189195 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:33:45.192084 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:45.192065 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 04 11:33:45.193212 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:45.193076 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 04 11:33:45.193212 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:45.193111 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 04 11:33:45.193212 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:45.193166 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zp2md\"" Aug 04 11:33:45.193212 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:45.193198 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 04 11:33:45.193498 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:45.193361 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9rm6g\"" Aug 04 11:33:45.360685 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:45.360651 2570 generic.go:356] "Generic (PLEG): container finished" podID="ad3565b1-c324-4d62-88d6-274e04aaf220" containerID="a9d0a71ee9abc86cbf76e0079c4d374198b7a0cf9d805cf47cdcd2b2daee5717" exitCode=0 Aug 04 11:33:45.361088 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:45.360716 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t6mz2" event={"ID":"ad3565b1-c324-4d62-88d6-274e04aaf220","Type":"ContainerDied","Data":"a9d0a71ee9abc86cbf76e0079c4d374198b7a0cf9d805cf47cdcd2b2daee5717"} Aug 04 11:33:45.433287 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:45.433262 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84257250-96bb-4afa-91fa-f2f031eaa074-metrics-tls\") pod \"dns-default-pjxbb\" (UID: \"84257250-96bb-4afa-91fa-f2f031eaa074\") " pod="openshift-dns/dns-default-pjxbb" Aug 04 11:33:45.433399 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:45.433351 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:45.433492 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:45.433469 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 11:33:45.433541 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:45.433489 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 11:33:45.433541 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:45.433502 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-675cff8469-qlt5l: secret "image-registry-tls" not found Aug 04 11:33:45.433639 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:45.433543 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84257250-96bb-4afa-91fa-f2f031eaa074-metrics-tls podName:84257250-96bb-4afa-91fa-f2f031eaa074 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:47.43352246 +0000 UTC m=+39.678241987 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/84257250-96bb-4afa-91fa-f2f031eaa074-metrics-tls") pod "dns-default-pjxbb" (UID: "84257250-96bb-4afa-91fa-f2f031eaa074") : secret "dns-default-metrics-tls" not found Aug 04 11:33:45.433639 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:45.433564 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls podName:2b083227-6cfe-42a3-a2a6-14f733c5e5d2 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:47.433555025 +0000 UTC m=+39.678274545 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls") pod "image-registry-675cff8469-qlt5l" (UID: "2b083227-6cfe-42a3-a2a6-14f733c5e5d2") : secret "image-registry-tls" not found Aug 04 11:33:46.367817 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:46.367773 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t6mz2" event={"ID":"ad3565b1-c324-4d62-88d6-274e04aaf220","Type":"ContainerStarted","Data":"504594cf1a41d77fb5d81361ad63d414550d06c3644e9e00ac9d2bf0ed216fef"} Aug 04 11:33:46.397549 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:46.397499 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-t6mz2" podStartSLOduration=9.354112473 podStartE2EDuration="38.39748321s" totalImagesPullingTime="29.043370737s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:08 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:46.395747532 +0000 UTC m=+38.640467078" watchObservedRunningTime="2026-08-04 11:33:46.39748321 +0000 UTC m=+38.642202753" Aug 04 11:33:47.372039 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:47.371799 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-db4rq" event={"ID":"7e7bd05e-d544-4ac5-b099-612ea158fc36","Type":"ContainerStarted","Data":"bae36ecd46f21e1d2e88dc3ae1002393cdc144648a56ce0939c9a7288ef377c1"} Aug 04 11:33:47.390114 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:47.390074 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-db4rq" podStartSLOduration=1.74043522 podStartE2EDuration="4.39006147s" totalImagesPullingTime="2.64962625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:33:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:33:47.389636215 +0000 UTC m=+39.634355756" watchObservedRunningTime="2026-08-04 11:33:47.39006147 +0000 UTC m=+39.634781012" Aug 04 11:33:47.450078 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:47.450048 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:47.450196 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:47.450101 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84257250-96bb-4afa-91fa-f2f031eaa074-metrics-tls\") pod \"dns-default-pjxbb\" (UID: \"84257250-96bb-4afa-91fa-f2f031eaa074\") " pod="openshift-dns/dns-default-pjxbb" Aug 04 11:33:47.450236 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:47.450195 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 11:33:47.450236 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:47.450208 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 11:33:47.450303 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:47.450271 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84257250-96bb-4afa-91fa-f2f031eaa074-metrics-tls podName:84257250-96bb-4afa-91fa-f2f031eaa074 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:51.450255971 +0000 UTC m=+43.694975494 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/84257250-96bb-4afa-91fa-f2f031eaa074-metrics-tls") pod "dns-default-pjxbb" (UID: "84257250-96bb-4afa-91fa-f2f031eaa074") : secret "dns-default-metrics-tls" not found Aug 04 11:33:47.450303 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:47.450216 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-675cff8469-qlt5l: secret "image-registry-tls" not found Aug 04 11:33:47.450385 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:47.450335 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls podName:2b083227-6cfe-42a3-a2a6-14f733c5e5d2 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:51.450321346 +0000 UTC m=+43.695040866 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls") pod "image-registry-675cff8469-qlt5l" (UID: "2b083227-6cfe-42a3-a2a6-14f733c5e5d2") : secret "image-registry-tls" not found Aug 04 11:33:51.477805 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:51.477769 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:51.478258 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:51.477829 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84257250-96bb-4afa-91fa-f2f031eaa074-metrics-tls\") pod \"dns-default-pjxbb\" (UID: \"84257250-96bb-4afa-91fa-f2f031eaa074\") " pod="openshift-dns/dns-default-pjxbb" Aug 04 11:33:51.478258 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:51.477933 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 11:33:51.478258 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:51.477935 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 11:33:51.478258 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:51.477954 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-675cff8469-qlt5l: secret "image-registry-tls" not found Aug 04 11:33:51.478258 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:51.477990 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84257250-96bb-4afa-91fa-f2f031eaa074-metrics-tls podName:84257250-96bb-4afa-91fa-f2f031eaa074 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:59.477975071 +0000 UTC m=+51.722694594 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/84257250-96bb-4afa-91fa-f2f031eaa074-metrics-tls") pod "dns-default-pjxbb" (UID: "84257250-96bb-4afa-91fa-f2f031eaa074") : secret "dns-default-metrics-tls" not found Aug 04 11:33:51.478258 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:51.478004 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls podName:2b083227-6cfe-42a3-a2a6-14f733c5e5d2 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:59.477998623 +0000 UTC m=+51.722718143 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls") pod "image-registry-675cff8469-qlt5l" (UID: "2b083227-6cfe-42a3-a2a6-14f733c5e5d2") : secret "image-registry-tls" not found Aug 04 11:33:52.904079 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:52.904049 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-klkv6"] Aug 04 11:33:52.934795 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:52.934765 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-klkv6"] Aug 04 11:33:52.934918 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:52.934859 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:33:52.937219 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:52.937196 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Aug 04 11:33:52.937422 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:52.937404 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Aug 04 11:33:52.937627 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:52.937606 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Aug 04 11:33:52.937705 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:52.937636 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fqqnm\"" Aug 04 11:33:52.938405 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:52.938388 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Aug 04 11:33:53.089642 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:53.089608 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlnmh\" (UniqueName: \"kubernetes.io/projected/2e4855b1-d090-4d03-a7bc-a5fff928a162-kube-api-access-qlnmh\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:33:53.089778 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:53.089681 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2e4855b1-d090-4d03-a7bc-a5fff928a162-crio-socket\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:33:53.089778 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:53.089701 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e4855b1-d090-4d03-a7bc-a5fff928a162-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:33:53.089778 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:53.089724 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e4855b1-d090-4d03-a7bc-a5fff928a162-data-volume\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:33:53.089778 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:53.089738 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e4855b1-d090-4d03-a7bc-a5fff928a162-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:33:53.190907 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:53.190824 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2e4855b1-d090-4d03-a7bc-a5fff928a162-crio-socket\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:33:53.190907 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:53.190862 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e4855b1-d090-4d03-a7bc-a5fff928a162-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:33:53.190907 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:53.190894 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e4855b1-d090-4d03-a7bc-a5fff928a162-data-volume\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:33:53.191115 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:53.190915 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e4855b1-d090-4d03-a7bc-a5fff928a162-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:33:53.191115 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:53.190954 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlnmh\" (UniqueName: \"kubernetes.io/projected/2e4855b1-d090-4d03-a7bc-a5fff928a162-kube-api-access-qlnmh\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:33:53.191115 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:53.191008 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 11:33:53.191115 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:53.191057 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 11:33:53.191115 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:53.191067 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2e4855b1-d090-4d03-a7bc-a5fff928a162-crio-socket\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:33:53.191115 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:53.191114 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e4855b1-d090-4d03-a7bc-a5fff928a162-insights-runtime-extractor-tls podName:2e4855b1-d090-4d03-a7bc-a5fff928a162 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:53.691091891 +0000 UTC m=+45.935811412 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2e4855b1-d090-4d03-a7bc-a5fff928a162-insights-runtime-extractor-tls") pod "insights-runtime-extractor-klkv6" (UID: "2e4855b1-d090-4d03-a7bc-a5fff928a162") : secret "insights-runtime-extractor-tls" not found Aug 04 11:33:53.191305 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:53.191137 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e4855b1-d090-4d03-a7bc-a5fff928a162-kube-rbac-proxy-cm podName:2e4855b1-d090-4d03-a7bc-a5fff928a162 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:53.69112604 +0000 UTC m=+45.935845566 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2e4855b1-d090-4d03-a7bc-a5fff928a162-kube-rbac-proxy-cm") pod "insights-runtime-extractor-klkv6" (UID: "2e4855b1-d090-4d03-a7bc-a5fff928a162") : configmap "kube-rbac-proxy" not found Aug 04 11:33:53.191305 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:53.191254 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e4855b1-d090-4d03-a7bc-a5fff928a162-data-volume\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:33:53.203964 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:53.203946 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlnmh\" (UniqueName: \"kubernetes.io/projected/2e4855b1-d090-4d03-a7bc-a5fff928a162-kube-api-access-qlnmh\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:33:53.694177 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:53.694140 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e4855b1-d090-4d03-a7bc-a5fff928a162-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:33:53.694177 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:53.694184 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e4855b1-d090-4d03-a7bc-a5fff928a162-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:33:53.694381 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:53.694277 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 11:33:53.694381 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:53.694307 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 11:33:53.694381 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:53.694350 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e4855b1-d090-4d03-a7bc-a5fff928a162-kube-rbac-proxy-cm podName:2e4855b1-d090-4d03-a7bc-a5fff928a162 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:54.694329148 +0000 UTC m=+46.939048676 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2e4855b1-d090-4d03-a7bc-a5fff928a162-kube-rbac-proxy-cm") pod "insights-runtime-extractor-klkv6" (UID: "2e4855b1-d090-4d03-a7bc-a5fff928a162") : configmap "kube-rbac-proxy" not found Aug 04 11:33:53.694381 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:53.694367 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e4855b1-d090-4d03-a7bc-a5fff928a162-insights-runtime-extractor-tls podName:2e4855b1-d090-4d03-a7bc-a5fff928a162 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:54.694360186 +0000 UTC m=+46.939079706 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2e4855b1-d090-4d03-a7bc-a5fff928a162-insights-runtime-extractor-tls") pod "insights-runtime-extractor-klkv6" (UID: "2e4855b1-d090-4d03-a7bc-a5fff928a162") : secret "insights-runtime-extractor-tls" not found Aug 04 11:33:54.700792 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:54.700749 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e4855b1-d090-4d03-a7bc-a5fff928a162-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:33:54.700792 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:54.700800 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e4855b1-d090-4d03-a7bc-a5fff928a162-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:33:54.701202 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:54.700860 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 11:33:54.701202 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:54.700896 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 11:33:54.701202 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:54.700905 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e4855b1-d090-4d03-a7bc-a5fff928a162-kube-rbac-proxy-cm podName:2e4855b1-d090-4d03-a7bc-a5fff928a162 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:56.700892782 +0000 UTC m=+48.945612302 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2e4855b1-d090-4d03-a7bc-a5fff928a162-kube-rbac-proxy-cm") pod "insights-runtime-extractor-klkv6" (UID: "2e4855b1-d090-4d03-a7bc-a5fff928a162") : configmap "kube-rbac-proxy" not found Aug 04 11:33:54.701202 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:54.701046 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e4855b1-d090-4d03-a7bc-a5fff928a162-insights-runtime-extractor-tls podName:2e4855b1-d090-4d03-a7bc-a5fff928a162 nodeName:}" failed. No retries permitted until 2026-08-04 11:33:56.70102622 +0000 UTC m=+48.945745743 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2e4855b1-d090-4d03-a7bc-a5fff928a162-insights-runtime-extractor-tls") pod "insights-runtime-extractor-klkv6" (UID: "2e4855b1-d090-4d03-a7bc-a5fff928a162") : secret "insights-runtime-extractor-tls" not found Aug 04 11:33:54.876369 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:54.876345 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hbfpg_f8dbbbc0-29a4-4849-9700-f9f8ebb310d8/dns-node-resolver/0.log" Aug 04 11:33:55.675853 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:55.675827 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5z7cf_8f2acb16-9405-4d91-853d-6a54e7a03a59/node-ca/0.log" Aug 04 11:33:56.677161 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:56.677134 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-db4rq_7e7bd05e-d544-4ac5-b099-612ea158fc36/serve-healthcheck-canary/0.log" Aug 04 11:33:56.718501 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:56.718470 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e4855b1-d090-4d03-a7bc-a5fff928a162-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:33:56.718598 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:56.718517 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e4855b1-d090-4d03-a7bc-a5fff928a162-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:33:56.718598 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:56.718578 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 11:33:56.718680 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:56.718615 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 11:33:56.718680 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:56.718672 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e4855b1-d090-4d03-a7bc-a5fff928a162-insights-runtime-extractor-tls podName:2e4855b1-d090-4d03-a7bc-a5fff928a162 nodeName:}" failed. No retries permitted until 2026-08-04 11:34:00.718655122 +0000 UTC m=+52.963374644 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2e4855b1-d090-4d03-a7bc-a5fff928a162-insights-runtime-extractor-tls") pod "insights-runtime-extractor-klkv6" (UID: "2e4855b1-d090-4d03-a7bc-a5fff928a162") : secret "insights-runtime-extractor-tls" not found Aug 04 11:33:56.718760 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:56.718754 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e4855b1-d090-4d03-a7bc-a5fff928a162-kube-rbac-proxy-cm podName:2e4855b1-d090-4d03-a7bc-a5fff928a162 nodeName:}" failed. No retries permitted until 2026-08-04 11:34:00.71874754 +0000 UTC m=+52.963467059 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2e4855b1-d090-4d03-a7bc-a5fff928a162-kube-rbac-proxy-cm") pod "insights-runtime-extractor-klkv6" (UID: "2e4855b1-d090-4d03-a7bc-a5fff928a162") : configmap "kube-rbac-proxy" not found Aug 04 11:33:59.539271 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:59.539237 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84257250-96bb-4afa-91fa-f2f031eaa074-metrics-tls\") pod \"dns-default-pjxbb\" (UID: \"84257250-96bb-4afa-91fa-f2f031eaa074\") " pod="openshift-dns/dns-default-pjxbb" Aug 04 11:33:59.539668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:33:59.539317 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:33:59.539668 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:59.539396 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 11:33:59.539668 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:59.539422 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 11:33:59.539668 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:59.539432 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-675cff8469-qlt5l: secret "image-registry-tls" not found Aug 04 11:33:59.539668 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:59.539484 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84257250-96bb-4afa-91fa-f2f031eaa074-metrics-tls podName:84257250-96bb-4afa-91fa-f2f031eaa074 nodeName:}" failed. No retries permitted until 2026-08-04 11:34:15.539463922 +0000 UTC m=+67.784183456 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/84257250-96bb-4afa-91fa-f2f031eaa074-metrics-tls") pod "dns-default-pjxbb" (UID: "84257250-96bb-4afa-91fa-f2f031eaa074") : secret "dns-default-metrics-tls" not found Aug 04 11:33:59.539668 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:33:59.539498 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls podName:2b083227-6cfe-42a3-a2a6-14f733c5e5d2 nodeName:}" failed. No retries permitted until 2026-08-04 11:34:15.53949208 +0000 UTC m=+67.784211599 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls") pod "image-registry-675cff8469-qlt5l" (UID: "2b083227-6cfe-42a3-a2a6-14f733c5e5d2") : secret "image-registry-tls" not found Aug 04 11:34:00.748568 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:00.748531 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e4855b1-d090-4d03-a7bc-a5fff928a162-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:34:00.748568 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:00.748571 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e4855b1-d090-4d03-a7bc-a5fff928a162-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:34:00.748994 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:34:00.748677 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 11:34:00.748994 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:34:00.748742 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e4855b1-d090-4d03-a7bc-a5fff928a162-insights-runtime-extractor-tls podName:2e4855b1-d090-4d03-a7bc-a5fff928a162 nodeName:}" failed. No retries permitted until 2026-08-04 11:34:08.748724103 +0000 UTC m=+60.993443630 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2e4855b1-d090-4d03-a7bc-a5fff928a162-insights-runtime-extractor-tls") pod "insights-runtime-extractor-klkv6" (UID: "2e4855b1-d090-4d03-a7bc-a5fff928a162") : secret "insights-runtime-extractor-tls" not found Aug 04 11:34:00.749068 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:00.749046 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e4855b1-d090-4d03-a7bc-a5fff928a162-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:34:08.807524 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:08.807483 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e4855b1-d090-4d03-a7bc-a5fff928a162-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:34:08.809697 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:08.809672 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e4855b1-d090-4d03-a7bc-a5fff928a162-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-klkv6\" (UID: \"2e4855b1-d090-4d03-a7bc-a5fff928a162\") " pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:34:08.846530 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:08.846509 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fqqnm\"" Aug 04 11:34:08.854443 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:08.854425 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-klkv6" Aug 04 11:34:09.006926 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:09.006883 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-klkv6"] Aug 04 11:34:09.025220 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:34:09.025194 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e4855b1_d090_4d03_a7bc_a5fff928a162.slice/crio-eece8e90ac9a79ee8128d5048edb4b2cfa7e834d9a886f59c2a70d235635084c WatchSource:0}: Error finding container eece8e90ac9a79ee8128d5048edb4b2cfa7e834d9a886f59c2a70d235635084c: Status 404 returned error can't find the container with id eece8e90ac9a79ee8128d5048edb4b2cfa7e834d9a886f59c2a70d235635084c Aug 04 11:34:09.419566 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:09.419490 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-klkv6" event={"ID":"2e4855b1-d090-4d03-a7bc-a5fff928a162","Type":"ContainerStarted","Data":"3cbc390bef65ed1f9b6071f29b45e35b0be79811fe803b196ced2c3bb1ea5a4f"} Aug 04 11:34:09.419566 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:09.419546 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-klkv6" event={"ID":"2e4855b1-d090-4d03-a7bc-a5fff928a162","Type":"ContainerStarted","Data":"eece8e90ac9a79ee8128d5048edb4b2cfa7e834d9a886f59c2a70d235635084c"} Aug 04 11:34:10.354630 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:10.354598 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-88j8c" Aug 04 11:34:10.424436 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:10.424400 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-klkv6" event={"ID":"2e4855b1-d090-4d03-a7bc-a5fff928a162","Type":"ContainerStarted","Data":"9ab45ef781e228197af7eb8edc3b50a49649a0006e8c26c5ab0afc71cdce6b83"} Aug 04 11:34:11.428425 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:11.428340 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-klkv6" event={"ID":"2e4855b1-d090-4d03-a7bc-a5fff928a162","Type":"ContainerStarted","Data":"0f54cb986125e5af5b6b05258459f5239f3d918298dd5e0a213fc397dd8c3807"} Aug 04 11:34:11.447388 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:11.447345 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-klkv6" podStartSLOduration=17.585741015 podStartE2EDuration="19.447333031s" totalImagesPullingTime="1.861592016s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:11.44582068 +0000 UTC m=+63.690540227" watchObservedRunningTime="2026-08-04 11:34:11.447333031 +0000 UTC m=+63.692052572" Aug 04 11:34:13.842387 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:13.842334 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret\") pod \"global-pull-secret-syncer-lcqpb\" (UID: \"e3158842-36a1-4213-af82-c6db89481715\") " pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:34:13.845251 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:13.845233 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 04 11:34:13.854773 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:13.854752 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3158842-36a1-4213-af82-c6db89481715-original-pull-secret\") pod \"global-pull-secret-syncer-lcqpb\" (UID: \"e3158842-36a1-4213-af82-c6db89481715\") " pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:34:13.943170 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:13.943121 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs\") pod \"network-metrics-daemon-kt4m7\" (UID: \"fe775771-24c4-46a8-b53a-4322c76ec540\") " pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:34:13.946123 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:13.946102 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 04 11:34:13.955570 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:13.955548 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe775771-24c4-46a8-b53a-4322c76ec540-metrics-certs\") pod \"network-metrics-daemon-kt4m7\" (UID: \"fe775771-24c4-46a8-b53a-4322c76ec540\") " pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:34:14.011111 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.011085 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9rm6g\"" Aug 04 11:34:14.015342 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.015325 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lcqpb" Aug 04 11:34:14.019236 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.019221 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kt4m7" Aug 04 11:34:14.044549 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.044522 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w2l4\" (UniqueName: \"kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4\") pod \"network-check-target-42v7s\" (UID: \"f1f75655-126a-410b-ab43-16e96cc67d05\") " pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:34:14.047433 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.047408 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 04 11:34:14.057418 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.057399 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 04 11:34:14.068652 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.068597 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4w2l4\" (UniqueName: \"kubernetes.io/projected/f1f75655-126a-410b-ab43-16e96cc67d05-kube-api-access-4w2l4\") pod \"network-check-target-42v7s\" (UID: \"f1f75655-126a-410b-ab43-16e96cc67d05\") " pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:34:14.160951 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.160921 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lcqpb"] Aug 04 11:34:14.164167 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:34:14.164141 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3158842_36a1_4213_af82_c6db89481715.slice/crio-40d6ec622fbd59a4b203c5ee9024ecd8d64111b68bdc136065aa9e8544b8e449 WatchSource:0}: Error finding container 40d6ec622fbd59a4b203c5ee9024ecd8d64111b68bdc136065aa9e8544b8e449: Status 404 returned error can't find the container with id 40d6ec622fbd59a4b203c5ee9024ecd8d64111b68bdc136065aa9e8544b8e449 Aug 04 11:34:14.176041 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.176010 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kt4m7"] Aug 04 11:34:14.179281 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:34:14.179260 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe775771_24c4_46a8_b53a_4322c76ec540.slice/crio-7ce2aeec54c0ea00a31bd3020e2150b090767dd3ae6c395d7a9b079399921c5d WatchSource:0}: Error finding container 7ce2aeec54c0ea00a31bd3020e2150b090767dd3ae6c395d7a9b079399921c5d: Status 404 returned error can't find the container with id 7ce2aeec54c0ea00a31bd3020e2150b090767dd3ae6c395d7a9b079399921c5d Aug 04 11:34:14.302955 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.302925 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zp2md\"" Aug 04 11:34:14.311826 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.311808 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:34:14.436055 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.435987 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kt4m7" event={"ID":"fe775771-24c4-46a8-b53a-4322c76ec540","Type":"ContainerStarted","Data":"7ce2aeec54c0ea00a31bd3020e2150b090767dd3ae6c395d7a9b079399921c5d"} Aug 04 11:34:14.437211 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.437189 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-42v7s"] Aug 04 11:34:14.437345 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.437326 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lcqpb" event={"ID":"e3158842-36a1-4213-af82-c6db89481715","Type":"ContainerStarted","Data":"40d6ec622fbd59a4b203c5ee9024ecd8d64111b68bdc136065aa9e8544b8e449"} Aug 04 11:34:14.440127 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:34:14.440106 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1f75655_126a_410b_ab43_16e96cc67d05.slice/crio-0f4f11edb22b1c884c141916275537178f397d3b22989f66ba10da86e445b315 WatchSource:0}: Error finding container 0f4f11edb22b1c884c141916275537178f397d3b22989f66ba10da86e445b315: Status 404 returned error can't find the container with id 0f4f11edb22b1c884c141916275537178f397d3b22989f66ba10da86e445b315 Aug 04 11:34:14.542368 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.542282 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cd6ff9d8-kp6bc"] Aug 04 11:34:14.545489 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.545468 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cd6ff9d8-kp6bc" Aug 04 11:34:14.547960 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.547932 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-859d6c4557-85cfq"] Aug 04 11:34:14.548087 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.548067 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Aug 04 11:34:14.548160 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.548140 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Aug 04 11:34:14.549158 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.549131 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Aug 04 11:34:14.549272 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.549141 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Aug 04 11:34:14.549337 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.549150 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-gjxd7\"" Aug 04 11:34:14.550973 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.550834 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859d6c4557-85cfq" Aug 04 11:34:14.553365 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.553204 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Aug 04 11:34:14.566685 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.566638 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cd6ff9d8-kp6bc"] Aug 04 11:34:14.567001 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.566967 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-859d6c4557-85cfq"] Aug 04 11:34:14.614868 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.614829 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-675cff8469-qlt5l"] Aug 04 11:34:14.615116 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:34:14.615070 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-675cff8469-qlt5l" podUID="2b083227-6cfe-42a3-a2a6-14f733c5e5d2" Aug 04 11:34:14.649732 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.649653 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d00188b3-4283-423d-8ed2-a544f60a9fe0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6cd6ff9d8-kp6bc\" (UID: \"d00188b3-4283-423d-8ed2-a544f60a9fe0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cd6ff9d8-kp6bc" Aug 04 11:34:14.649732 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.649690 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ca13501-19f3-4e9f-b129-8915a1756293-tmp\") pod \"klusterlet-addon-workmgr-859d6c4557-85cfq\" (UID: \"1ca13501-19f3-4e9f-b129-8915a1756293\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859d6c4557-85cfq" Aug 04 11:34:14.649927 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.649779 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh7dx\" (UniqueName: \"kubernetes.io/projected/d00188b3-4283-423d-8ed2-a544f60a9fe0-kube-api-access-wh7dx\") pod \"managed-serviceaccount-addon-agent-6cd6ff9d8-kp6bc\" (UID: \"d00188b3-4283-423d-8ed2-a544f60a9fe0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cd6ff9d8-kp6bc" Aug 04 11:34:14.649927 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.649868 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsck7\" (UniqueName: \"kubernetes.io/projected/1ca13501-19f3-4e9f-b129-8915a1756293-kube-api-access-gsck7\") pod \"klusterlet-addon-workmgr-859d6c4557-85cfq\" (UID: \"1ca13501-19f3-4e9f-b129-8915a1756293\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859d6c4557-85cfq" Aug 04 11:34:14.649927 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.649906 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1ca13501-19f3-4e9f-b129-8915a1756293-klusterlet-config\") pod \"klusterlet-addon-workmgr-859d6c4557-85cfq\" (UID: \"1ca13501-19f3-4e9f-b129-8915a1756293\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859d6c4557-85cfq" Aug 04 11:34:14.683848 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.683811 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-789fcb8b4c-mqrqd"] Aug 04 11:34:14.686967 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.686947 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.707192 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.707132 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-789fcb8b4c-mqrqd"] Aug 04 11:34:14.750348 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.750307 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d00188b3-4283-423d-8ed2-a544f60a9fe0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6cd6ff9d8-kp6bc\" (UID: \"d00188b3-4283-423d-8ed2-a544f60a9fe0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cd6ff9d8-kp6bc" Aug 04 11:34:14.750541 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.750361 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdqz5\" (UniqueName: \"kubernetes.io/projected/f2fef570-6051-45ae-9317-592c2708fa0f-kube-api-access-bdqz5\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.750541 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.750391 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2fef570-6051-45ae-9317-592c2708fa0f-registry-tls\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.750541 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.750457 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f2fef570-6051-45ae-9317-592c2708fa0f-installation-pull-secrets\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.750541 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.750491 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2fef570-6051-45ae-9317-592c2708fa0f-bound-sa-token\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.750541 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.750525 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1ca13501-19f3-4e9f-b129-8915a1756293-klusterlet-config\") pod \"klusterlet-addon-workmgr-859d6c4557-85cfq\" (UID: \"1ca13501-19f3-4e9f-b129-8915a1756293\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859d6c4557-85cfq" Aug 04 11:34:14.750734 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.750592 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ca13501-19f3-4e9f-b129-8915a1756293-tmp\") pod \"klusterlet-addon-workmgr-859d6c4557-85cfq\" (UID: \"1ca13501-19f3-4e9f-b129-8915a1756293\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859d6c4557-85cfq" Aug 04 11:34:14.750734 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.750610 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh7dx\" (UniqueName: \"kubernetes.io/projected/d00188b3-4283-423d-8ed2-a544f60a9fe0-kube-api-access-wh7dx\") pod \"managed-serviceaccount-addon-agent-6cd6ff9d8-kp6bc\" (UID: \"d00188b3-4283-423d-8ed2-a544f60a9fe0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cd6ff9d8-kp6bc" Aug 04 11:34:14.750734 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.750634 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f2fef570-6051-45ae-9317-592c2708fa0f-registry-certificates\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.750734 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.750654 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsck7\" (UniqueName: \"kubernetes.io/projected/1ca13501-19f3-4e9f-b129-8915a1756293-kube-api-access-gsck7\") pod \"klusterlet-addon-workmgr-859d6c4557-85cfq\" (UID: \"1ca13501-19f3-4e9f-b129-8915a1756293\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859d6c4557-85cfq" Aug 04 11:34:14.750734 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.750679 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f2fef570-6051-45ae-9317-592c2708fa0f-image-registry-private-configuration\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.750734 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.750697 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f2fef570-6051-45ae-9317-592c2708fa0f-trusted-ca\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.750734 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.750716 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f2fef570-6051-45ae-9317-592c2708fa0f-ca-trust-extracted\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.751177 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.751094 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ca13501-19f3-4e9f-b129-8915a1756293-tmp\") pod \"klusterlet-addon-workmgr-859d6c4557-85cfq\" (UID: \"1ca13501-19f3-4e9f-b129-8915a1756293\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859d6c4557-85cfq" Aug 04 11:34:14.753639 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.753595 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1ca13501-19f3-4e9f-b129-8915a1756293-klusterlet-config\") pod \"klusterlet-addon-workmgr-859d6c4557-85cfq\" (UID: \"1ca13501-19f3-4e9f-b129-8915a1756293\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859d6c4557-85cfq" Aug 04 11:34:14.753639 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.753629 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d00188b3-4283-423d-8ed2-a544f60a9fe0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6cd6ff9d8-kp6bc\" (UID: \"d00188b3-4283-423d-8ed2-a544f60a9fe0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cd6ff9d8-kp6bc" Aug 04 11:34:14.784413 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.784379 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsck7\" (UniqueName: \"kubernetes.io/projected/1ca13501-19f3-4e9f-b129-8915a1756293-kube-api-access-gsck7\") pod \"klusterlet-addon-workmgr-859d6c4557-85cfq\" (UID: \"1ca13501-19f3-4e9f-b129-8915a1756293\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859d6c4557-85cfq" Aug 04 11:34:14.786510 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.786486 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh7dx\" (UniqueName: \"kubernetes.io/projected/d00188b3-4283-423d-8ed2-a544f60a9fe0-kube-api-access-wh7dx\") pod \"managed-serviceaccount-addon-agent-6cd6ff9d8-kp6bc\" (UID: \"d00188b3-4283-423d-8ed2-a544f60a9fe0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cd6ff9d8-kp6bc" Aug 04 11:34:14.851601 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.851566 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f2fef570-6051-45ae-9317-592c2708fa0f-installation-pull-secrets\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.851601 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.851608 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2fef570-6051-45ae-9317-592c2708fa0f-bound-sa-token\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.852071 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.851675 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f2fef570-6051-45ae-9317-592c2708fa0f-registry-certificates\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.852071 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.851701 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f2fef570-6051-45ae-9317-592c2708fa0f-image-registry-private-configuration\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.852071 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.851722 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f2fef570-6051-45ae-9317-592c2708fa0f-trusted-ca\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.852210 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.852146 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f2fef570-6051-45ae-9317-592c2708fa0f-ca-trust-extracted\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.852210 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.852195 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdqz5\" (UniqueName: \"kubernetes.io/projected/f2fef570-6051-45ae-9317-592c2708fa0f-kube-api-access-bdqz5\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.852293 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.852223 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2fef570-6051-45ae-9317-592c2708fa0f-registry-tls\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.852779 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.852751 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f2fef570-6051-45ae-9317-592c2708fa0f-registry-certificates\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.852898 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.852782 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f2fef570-6051-45ae-9317-592c2708fa0f-trusted-ca\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.852964 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.852941 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f2fef570-6051-45ae-9317-592c2708fa0f-ca-trust-extracted\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.854571 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.854509 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f2fef570-6051-45ae-9317-592c2708fa0f-installation-pull-secrets\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.855023 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.854991 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f2fef570-6051-45ae-9317-592c2708fa0f-image-registry-private-configuration\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.855220 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.855199 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2fef570-6051-45ae-9317-592c2708fa0f-registry-tls\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.866088 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.866059 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdqz5\" (UniqueName: \"kubernetes.io/projected/f2fef570-6051-45ae-9317-592c2708fa0f-kube-api-access-bdqz5\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.866182 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.866162 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cd6ff9d8-kp6bc" Aug 04 11:34:14.866608 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.866587 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2fef570-6051-45ae-9317-592c2708fa0f-bound-sa-token\") pod \"image-registry-789fcb8b4c-mqrqd\" (UID: \"f2fef570-6051-45ae-9317-592c2708fa0f\") " pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:14.873661 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:14.873493 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859d6c4557-85cfq" Aug 04 11:34:15.000813 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.000761 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gql25\"" Aug 04 11:34:15.008561 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.008531 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:15.260997 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.260963 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-789fcb8b4c-mqrqd"] Aug 04 11:34:15.265007 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:34:15.264939 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2fef570_6051_45ae_9317_592c2708fa0f.slice/crio-729e763ed85c01d8c77aa9d50abff7dfb0c3eb06f1676275c9c7defeb0c747de WatchSource:0}: Error finding container 729e763ed85c01d8c77aa9d50abff7dfb0c3eb06f1676275c9c7defeb0c747de: Status 404 returned error can't find the container with id 729e763ed85c01d8c77aa9d50abff7dfb0c3eb06f1676275c9c7defeb0c747de Aug 04 11:34:15.277492 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.277429 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cd6ff9d8-kp6bc"] Aug 04 11:34:15.280967 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:34:15.280940 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd00188b3_4283_423d_8ed2_a544f60a9fe0.slice/crio-6f6cd2b6caec9b8cadc81e44ae06e239ea56040285c8623d5833b15b0aa68bc4 WatchSource:0}: Error finding container 6f6cd2b6caec9b8cadc81e44ae06e239ea56040285c8623d5833b15b0aa68bc4: Status 404 returned error can't find the container with id 6f6cd2b6caec9b8cadc81e44ae06e239ea56040285c8623d5833b15b0aa68bc4 Aug 04 11:34:15.445366 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.445328 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cd6ff9d8-kp6bc" event={"ID":"d00188b3-4283-423d-8ed2-a544f60a9fe0","Type":"ContainerStarted","Data":"6f6cd2b6caec9b8cadc81e44ae06e239ea56040285c8623d5833b15b0aa68bc4"} Aug 04 11:34:15.447077 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.447047 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" event={"ID":"f2fef570-6051-45ae-9317-592c2708fa0f","Type":"ContainerStarted","Data":"a3c447a97796524b8247afe31f3c98d464d30f22ab6d0239dfae973ff5d3dc17"} Aug 04 11:34:15.447142 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.447109 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" event={"ID":"f2fef570-6051-45ae-9317-592c2708fa0f","Type":"ContainerStarted","Data":"729e763ed85c01d8c77aa9d50abff7dfb0c3eb06f1676275c9c7defeb0c747de"} Aug 04 11:34:15.447194 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.447154 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:15.448542 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.448472 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-42v7s" event={"ID":"f1f75655-126a-410b-ab43-16e96cc67d05","Type":"ContainerStarted","Data":"0f4f11edb22b1c884c141916275537178f397d3b22989f66ba10da86e445b315"} Aug 04 11:34:15.450255 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.450028 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:34:15.450355 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.450255 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kt4m7" event={"ID":"fe775771-24c4-46a8-b53a-4322c76ec540","Type":"ContainerStarted","Data":"60be94d1d70dd42fd3fafc1553e70fb9b82e575480da1651a1bf5d12c18cc3f6"} Aug 04 11:34:15.473902 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.473763 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" podStartSLOduration=1.473747384 podStartE2EDuration="1.473747384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:34:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:15.472525776 +0000 UTC m=+67.717245317" watchObservedRunningTime="2026-08-04 11:34:15.473747384 +0000 UTC m=+67.718466925" Aug 04 11:34:15.486315 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.486194 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:34:15.490875 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.490853 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-859d6c4557-85cfq"] Aug 04 11:34:15.495399 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:34:15.495374 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ca13501_19f3_4e9f_b129_8915a1756293.slice/crio-b05b28cde5079accac4a4839ad5f3a4e41335f1420acb502850da094f2b48e17 WatchSource:0}: Error finding container b05b28cde5079accac4a4839ad5f3a4e41335f1420acb502850da094f2b48e17: Status 404 returned error can't find the container with id b05b28cde5079accac4a4839ad5f3a4e41335f1420acb502850da094f2b48e17 Aug 04 11:34:15.561097 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.560590 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-bound-sa-token\") pod \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " Aug 04 11:34:15.561097 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.560643 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlcpr\" (UniqueName: \"kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-kube-api-access-jlcpr\") pod \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " Aug 04 11:34:15.561097 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.560700 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-installation-pull-secrets\") pod \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " Aug 04 11:34:15.561097 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.560888 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-certificates\") pod \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " Aug 04 11:34:15.561097 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.560939 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-ca-trust-extracted\") pod \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " Aug 04 11:34:15.561097 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.560971 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-image-registry-private-configuration\") pod \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " Aug 04 11:34:15.561097 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.560999 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-trusted-ca\") pod \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " Aug 04 11:34:15.561917 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.561181 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84257250-96bb-4afa-91fa-f2f031eaa074-metrics-tls\") pod \"dns-default-pjxbb\" (UID: \"84257250-96bb-4afa-91fa-f2f031eaa074\") " pod="openshift-dns/dns-default-pjxbb" Aug 04 11:34:15.561917 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.561262 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:34:15.561917 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.561537 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2b083227-6cfe-42a3-a2a6-14f733c5e5d2" (UID: "2b083227-6cfe-42a3-a2a6-14f733c5e5d2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 11:34:15.563093 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.562101 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2b083227-6cfe-42a3-a2a6-14f733c5e5d2" (UID: "2b083227-6cfe-42a3-a2a6-14f733c5e5d2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:34:15.563093 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.562681 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2b083227-6cfe-42a3-a2a6-14f733c5e5d2" (UID: "2b083227-6cfe-42a3-a2a6-14f733c5e5d2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:34:15.563834 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.563792 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-kube-api-access-jlcpr" (OuterVolumeSpecName: "kube-api-access-jlcpr") pod "2b083227-6cfe-42a3-a2a6-14f733c5e5d2" (UID: "2b083227-6cfe-42a3-a2a6-14f733c5e5d2"). InnerVolumeSpecName "kube-api-access-jlcpr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:34:15.564539 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.564504 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2b083227-6cfe-42a3-a2a6-14f733c5e5d2" (UID: "2b083227-6cfe-42a3-a2a6-14f733c5e5d2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:34:15.564634 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.564531 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2b083227-6cfe-42a3-a2a6-14f733c5e5d2" (UID: "2b083227-6cfe-42a3-a2a6-14f733c5e5d2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 11:34:15.564634 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.564545 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls\") pod \"image-registry-675cff8469-qlt5l\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:34:15.565120 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.565100 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2b083227-6cfe-42a3-a2a6-14f733c5e5d2" (UID: "2b083227-6cfe-42a3-a2a6-14f733c5e5d2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 11:34:15.565750 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.565725 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/84257250-96bb-4afa-91fa-f2f031eaa074-metrics-tls\") pod \"dns-default-pjxbb\" (UID: \"84257250-96bb-4afa-91fa-f2f031eaa074\") " pod="openshift-dns/dns-default-pjxbb" Aug 04 11:34:15.662088 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.662048 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls\") pod \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\" (UID: \"2b083227-6cfe-42a3-a2a6-14f733c5e5d2\") " Aug 04 11:34:15.662252 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.662234 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlcpr\" (UniqueName: \"kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-kube-api-access-jlcpr\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:34:15.662302 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.662252 2570 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-installation-pull-secrets\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:34:15.662302 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.662267 2570 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-certificates\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:34:15.662302 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.662281 2570 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-ca-trust-extracted\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:34:15.662302 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.662296 2570 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-image-registry-private-configuration\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:34:15.662532 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.662309 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-trusted-ca\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:34:15.662532 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.662324 2570 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-bound-sa-token\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:34:15.664324 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.664291 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2b083227-6cfe-42a3-a2a6-14f733c5e5d2" (UID: "2b083227-6cfe-42a3-a2a6-14f733c5e5d2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:34:15.763026 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.762986 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2b083227-6cfe-42a3-a2a6-14f733c5e5d2-registry-tls\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:34:15.828361 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.828286 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hll8p\"" Aug 04 11:34:15.836482 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:15.836368 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pjxbb" Aug 04 11:34:16.024887 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:16.024855 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pjxbb"] Aug 04 11:34:16.029063 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:34:16.029034 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84257250_96bb_4afa_91fa_f2f031eaa074.slice/crio-08a133eaabcaff1b452bf2fabfc608fc9228c04d3f2ade8c7453da18a4953ff8 WatchSource:0}: Error finding container 08a133eaabcaff1b452bf2fabfc608fc9228c04d3f2ade8c7453da18a4953ff8: Status 404 returned error can't find the container with id 08a133eaabcaff1b452bf2fabfc608fc9228c04d3f2ade8c7453da18a4953ff8 Aug 04 11:34:16.467576 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:16.467003 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pjxbb" event={"ID":"84257250-96bb-4afa-91fa-f2f031eaa074","Type":"ContainerStarted","Data":"08a133eaabcaff1b452bf2fabfc608fc9228c04d3f2ade8c7453da18a4953ff8"} Aug 04 11:34:16.471096 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:16.471067 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kt4m7" event={"ID":"fe775771-24c4-46a8-b53a-4322c76ec540","Type":"ContainerStarted","Data":"4733f6af3a6cb0398467fe08ca539204917ed5c560cddff0c7a0b9dca0fa9fc3"} Aug 04 11:34:16.473596 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:16.473039 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859d6c4557-85cfq" event={"ID":"1ca13501-19f3-4e9f-b129-8915a1756293","Type":"ContainerStarted","Data":"b05b28cde5079accac4a4839ad5f3a4e41335f1420acb502850da094f2b48e17"} Aug 04 11:34:16.473596 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:16.473113 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-675cff8469-qlt5l" Aug 04 11:34:16.488369 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:16.488274 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kt4m7" podStartSLOduration=67.515950929 podStartE2EDuration="1m8.488253722s" totalImagesPullingTime="972.302793ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:33:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:16.487048281 +0000 UTC m=+68.731767847" watchObservedRunningTime="2026-08-04 11:34:16.488253722 +0000 UTC m=+68.732973264" Aug 04 11:34:16.518261 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:16.518199 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-675cff8469-qlt5l"] Aug 04 11:34:16.521845 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:16.521782 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-675cff8469-qlt5l"] Aug 04 11:34:18.194199 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:18.194155 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b083227-6cfe-42a3-a2a6-14f733c5e5d2" path="/var/lib/kubelet/pods/2b083227-6cfe-42a3-a2a6-14f733c5e5d2/volumes" Aug 04 11:34:19.755500 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:19.755467 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-9459q"] Aug 04 11:34:19.757509 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:19.757491 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-9459q" Aug 04 11:34:19.759844 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:19.759817 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Aug 04 11:34:19.759956 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:19.759919 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-nh6qt\"" Aug 04 11:34:19.768385 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:19.768364 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-9459q"] Aug 04 11:34:19.792780 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:19.792754 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/775c79ad-948e-4fde-9707-7ebb75bb9981-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-9459q\" (UID: \"775c79ad-948e-4fde-9707-7ebb75bb9981\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-9459q" Aug 04 11:34:19.894076 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:19.894042 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/775c79ad-948e-4fde-9707-7ebb75bb9981-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-9459q\" (UID: \"775c79ad-948e-4fde-9707-7ebb75bb9981\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-9459q" Aug 04 11:34:19.894259 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:34:19.894196 2570 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Aug 04 11:34:19.894313 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:34:19.894274 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/775c79ad-948e-4fde-9707-7ebb75bb9981-tls-certificates podName:775c79ad-948e-4fde-9707-7ebb75bb9981 nodeName:}" failed. No retries permitted until 2026-08-04 11:34:20.394251611 +0000 UTC m=+72.638971132 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/775c79ad-948e-4fde-9707-7ebb75bb9981-tls-certificates") pod "prometheus-operator-admission-webhook-66685ff6d4-9459q" (UID: "775c79ad-948e-4fde-9707-7ebb75bb9981") : secret "prometheus-operator-admission-webhook-tls" not found Aug 04 11:34:20.397296 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:20.397256 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/775c79ad-948e-4fde-9707-7ebb75bb9981-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-9459q\" (UID: \"775c79ad-948e-4fde-9707-7ebb75bb9981\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-9459q" Aug 04 11:34:20.399842 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:20.399810 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/775c79ad-948e-4fde-9707-7ebb75bb9981-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-9459q\" (UID: \"775c79ad-948e-4fde-9707-7ebb75bb9981\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-9459q" Aug 04 11:34:20.667232 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:20.667156 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-9459q" Aug 04 11:34:22.928967 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:22.928849 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-9459q"] Aug 04 11:34:23.504979 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:23.504945 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-42v7s" event={"ID":"f1f75655-126a-410b-ab43-16e96cc67d05","Type":"ContainerStarted","Data":"048cb0ed2051c1235f8e56025eaa3a42558f4d07b55459f2994a3995d7d59021"} Aug 04 11:34:23.505154 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:23.505019 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:34:23.506603 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:23.506580 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pjxbb" event={"ID":"84257250-96bb-4afa-91fa-f2f031eaa074","Type":"ContainerStarted","Data":"63a7186418779621cb3bb2aaf685cd82f33914d16a87c0db46439f23d31e4d7e"} Aug 04 11:34:23.506679 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:23.506624 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pjxbb" event={"ID":"84257250-96bb-4afa-91fa-f2f031eaa074","Type":"ContainerStarted","Data":"9a9e44b9f9d02bc9ea75cb2389d0c81a734af6c55600c35469c3b503301724fc"} Aug 04 11:34:23.506679 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:23.506648 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pjxbb" Aug 04 11:34:23.507656 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:23.507635 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-9459q" event={"ID":"775c79ad-948e-4fde-9707-7ebb75bb9981","Type":"ContainerStarted","Data":"30e65688ceb61a83a76c6aef6fa77b1f4346bab52b526ac891552809c9379ef6"} Aug 04 11:34:23.508902 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:23.508882 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lcqpb" event={"ID":"e3158842-36a1-4213-af82-c6db89481715","Type":"ContainerStarted","Data":"a09c95e2c0a90ec205794f45fe9864824d034503aa29e0f2268d56ecf85bbc60"} Aug 04 11:34:23.510273 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:23.510254 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cd6ff9d8-kp6bc" event={"ID":"d00188b3-4283-423d-8ed2-a544f60a9fe0","Type":"ContainerStarted","Data":"370ae34d646634236884c9d655f8588bcb58f873aacabf174aee21e712904065"} Aug 04 11:34:23.511496 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:23.511476 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859d6c4557-85cfq" event={"ID":"1ca13501-19f3-4e9f-b129-8915a1756293","Type":"ContainerStarted","Data":"0a3dd7ff0f16a3ab9a99696246468334d78f397279eda5d0ae5cc7a53548818f"} Aug 04 11:34:23.511701 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:23.511688 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859d6c4557-85cfq" Aug 04 11:34:23.513203 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:23.513184 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859d6c4557-85cfq" Aug 04 11:34:23.522607 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:23.522571 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-42v7s" podStartSLOduration=67.195100195 podStartE2EDuration="1m15.522558919s" totalImagesPullingTime="8.327458724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:33:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:23.522372777 +0000 UTC m=+75.767092318" watchObservedRunningTime="2026-08-04 11:34:23.522558919 +0000 UTC m=+75.767278460" Aug 04 11:34:23.544972 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:23.544935 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pjxbb" podStartSLOduration=33.80831818 podStartE2EDuration="40.54492331s" totalImagesPullingTime="6.73660513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:33:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:23.544194597 +0000 UTC m=+75.788914152" watchObservedRunningTime="2026-08-04 11:34:23.54492331 +0000 UTC m=+75.789642851" Aug 04 11:34:23.561339 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:23.561297 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cd6ff9d8-kp6bc" podStartSLOduration=2.068723149 podStartE2EDuration="9.561285718s" totalImagesPullingTime="7.492562569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:34:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:23.560392821 +0000 UTC m=+75.805112357" watchObservedRunningTime="2026-08-04 11:34:23.561285718 +0000 UTC m=+75.806005307" Aug 04 11:34:23.582634 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:23.582594 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-859d6c4557-85cfq" podStartSLOduration=2.294688393 podStartE2EDuration="9.582583279s" totalImagesPullingTime="7.287894886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:34:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:23.582147758 +0000 UTC m=+75.826867325" watchObservedRunningTime="2026-08-04 11:34:23.582583279 +0000 UTC m=+75.827302820" Aug 04 11:34:23.602826 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:23.602789 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lcqpb" podStartSLOduration=66.999340066 podStartE2EDuration="1m15.602773966s" totalImagesPullingTime="8.6034339s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:33:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:23.602583572 +0000 UTC m=+75.847303114" watchObservedRunningTime="2026-08-04 11:34:23.602773966 +0000 UTC m=+75.847493509" Aug 04 11:34:24.516687 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:24.516596 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-9459q" event={"ID":"775c79ad-948e-4fde-9707-7ebb75bb9981","Type":"ContainerStarted","Data":"a89191d31e77b0bd4c1fbc69d569be9d5d720bf4038a2b1aef3b5d5288369748"} Aug 04 11:34:24.517130 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:24.517067 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-9459q" Aug 04 11:34:24.521621 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:24.521598 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-9459q" Aug 04 11:34:24.532688 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:24.532651 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-9459q" podStartSLOduration=4.216521119 podStartE2EDuration="5.532638838s" totalImagesPullingTime="1.316117719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:34:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:24.531569377 +0000 UTC m=+76.776288923" watchObservedRunningTime="2026-08-04 11:34:24.532638838 +0000 UTC m=+76.777358380" Aug 04 11:34:24.836119 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:24.836029 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-5xshl"] Aug 04 11:34:24.839158 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:24.839143 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-5xshl" Aug 04 11:34:24.842370 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:24.842348 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Aug 04 11:34:24.842649 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:24.842634 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Aug 04 11:34:24.842703 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:24.842672 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Aug 04 11:34:24.843932 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:24.843889 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Aug 04 11:34:24.843932 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:24.843907 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Aug 04 11:34:24.843932 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:24.843907 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-cgzlp\"" Aug 04 11:34:24.851787 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:24.851762 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-5xshl"] Aug 04 11:34:24.930024 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:24.929993 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-5xshl\" (UID: \"f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-5xshl" Aug 04 11:34:24.930180 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:24.930035 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-5xshl\" (UID: \"f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-5xshl" Aug 04 11:34:24.930180 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:24.930110 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-5xshl\" (UID: \"f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-5xshl" Aug 04 11:34:24.930180 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:24.930132 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsktd\" (UniqueName: \"kubernetes.io/projected/f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a-kube-api-access-hsktd\") pod \"prometheus-operator-758d7cb874-5xshl\" (UID: \"f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-5xshl" Aug 04 11:34:25.031247 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:25.031211 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-5xshl\" (UID: \"f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-5xshl" Aug 04 11:34:25.031247 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:25.031253 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-5xshl\" (UID: \"f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-5xshl" Aug 04 11:34:25.031434 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:25.031287 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-5xshl\" (UID: \"f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-5xshl" Aug 04 11:34:25.031434 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:25.031317 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsktd\" (UniqueName: \"kubernetes.io/projected/f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a-kube-api-access-hsktd\") pod \"prometheus-operator-758d7cb874-5xshl\" (UID: \"f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-5xshl" Aug 04 11:34:25.031525 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:34:25.031429 2570 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Aug 04 11:34:25.031525 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:34:25.031507 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a-prometheus-operator-tls podName:f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a nodeName:}" failed. No retries permitted until 2026-08-04 11:34:25.531490726 +0000 UTC m=+77.776210246 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a-prometheus-operator-tls") pod "prometheus-operator-758d7cb874-5xshl" (UID: "f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a") : secret "prometheus-operator-tls" not found Aug 04 11:34:25.031974 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:25.031958 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-5xshl\" (UID: \"f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-5xshl" Aug 04 11:34:25.033595 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:25.033573 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-5xshl\" (UID: \"f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-5xshl" Aug 04 11:34:25.039933 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:25.039901 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsktd\" (UniqueName: \"kubernetes.io/projected/f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a-kube-api-access-hsktd\") pod \"prometheus-operator-758d7cb874-5xshl\" (UID: \"f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-5xshl" Aug 04 11:34:25.534323 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:25.534296 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-5xshl\" (UID: \"f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-5xshl" Aug 04 11:34:25.536487 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:25.536469 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-5xshl\" (UID: \"f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-5xshl" Aug 04 11:34:25.747983 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:25.747937 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-5xshl" Aug 04 11:34:25.867719 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:25.867688 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-5xshl"] Aug 04 11:34:25.870273 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:34:25.870242 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf99581d4_f9ea_4f60_a6ce_a8f1fd03c14a.slice/crio-da101f803533164149d7b800d05100f2cbd8959fbdf2fd9df5723be575218740 WatchSource:0}: Error finding container da101f803533164149d7b800d05100f2cbd8959fbdf2fd9df5723be575218740: Status 404 returned error can't find the container with id da101f803533164149d7b800d05100f2cbd8959fbdf2fd9df5723be575218740 Aug 04 11:34:26.523941 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:26.523908 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-5xshl" event={"ID":"f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a","Type":"ContainerStarted","Data":"da101f803533164149d7b800d05100f2cbd8959fbdf2fd9df5723be575218740"} Aug 04 11:34:28.532792 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:28.532755 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-5xshl" event={"ID":"f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a","Type":"ContainerStarted","Data":"bf697a5b7607162dc7bd027b43786b382dc6f05cc66adbe85cf1c986fc808449"} Aug 04 11:34:28.533243 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:28.532810 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-5xshl" event={"ID":"f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a","Type":"ContainerStarted","Data":"723febdf825d200e37de1e04514c181829f28e7975b4ffe9f41fd3294b31ca5e"} Aug 04 11:34:28.551719 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:28.551668 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-758d7cb874-5xshl" podStartSLOduration=2.9342370190000002 podStartE2EDuration="4.551652817s" totalImagesPullingTime="1.617415798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:34:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:28.550497088 +0000 UTC m=+80.795216631" watchObservedRunningTime="2026-08-04 11:34:28.551652817 +0000 UTC m=+80.796372359" Aug 04 11:34:30.201283 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.201252 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bfqvz"] Aug 04 11:34:30.204739 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.204721 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.207550 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.207529 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Aug 04 11:34:30.207550 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.207539 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Aug 04 11:34:30.207905 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.207887 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Aug 04 11:34:30.208403 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.208386 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tqxrb\"" Aug 04 11:34:30.270299 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.270196 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-node-exporter-accelerators-collector-config\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.270299 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.270231 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-root\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.270299 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.270252 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-node-exporter-textfile\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.270514 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.270295 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-sys\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.270514 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.270334 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn6bp\" (UniqueName: \"kubernetes.io/projected/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-kube-api-access-pn6bp\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.270514 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.270392 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-node-exporter-tls\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.270514 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.270422 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.270514 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.270441 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-node-exporter-wtmp\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.270514 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.270481 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-metrics-client-ca\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.371689 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.371656 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-root\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.371689 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.371693 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-node-exporter-textfile\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.371870 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.371709 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-sys\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.371870 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.371735 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn6bp\" (UniqueName: \"kubernetes.io/projected/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-kube-api-access-pn6bp\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.371870 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.371761 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-root\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.371870 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.371775 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-node-exporter-tls\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.371870 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.371793 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-sys\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.371870 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.371824 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.371870 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.371857 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-node-exporter-wtmp\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.372201 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.371896 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-metrics-client-ca\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.372201 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.371936 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-node-exporter-accelerators-collector-config\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.372201 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.372010 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-node-exporter-wtmp\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.372201 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.372085 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-node-exporter-textfile\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.372492 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.372466 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-node-exporter-accelerators-collector-config\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.372570 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.372502 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-metrics-client-ca\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.374050 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.374023 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-node-exporter-tls\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.374149 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.374077 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.380117 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.380098 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn6bp\" (UniqueName: \"kubernetes.io/projected/fb8c73dd-6a50-4085-99ad-9ed552b4f3f3-kube-api-access-pn6bp\") pod \"node-exporter-bfqvz\" (UID: \"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3\") " pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.513344 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.513313 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bfqvz" Aug 04 11:34:30.524540 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:34:30.524514 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb8c73dd_6a50_4085_99ad_9ed552b4f3f3.slice/crio-310f15451cd6c34c4c8a5a551f1b2f2d139dc90d6ea90ca6c5c32cdca09885b0 WatchSource:0}: Error finding container 310f15451cd6c34c4c8a5a551f1b2f2d139dc90d6ea90ca6c5c32cdca09885b0: Status 404 returned error can't find the container with id 310f15451cd6c34c4c8a5a551f1b2f2d139dc90d6ea90ca6c5c32cdca09885b0 Aug 04 11:34:30.540668 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:30.540643 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bfqvz" event={"ID":"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3","Type":"ContainerStarted","Data":"310f15451cd6c34c4c8a5a551f1b2f2d139dc90d6ea90ca6c5c32cdca09885b0"} Aug 04 11:34:32.548827 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:32.548788 2570 generic.go:356] "Generic (PLEG): container finished" podID="fb8c73dd-6a50-4085-99ad-9ed552b4f3f3" containerID="2e07b5a92c735f2e8a03334946d6418478d8f42ff64395d847cd91c987cd4f7b" exitCode=0 Aug 04 11:34:32.548827 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:32.548827 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bfqvz" event={"ID":"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3","Type":"ContainerDied","Data":"2e07b5a92c735f2e8a03334946d6418478d8f42ff64395d847cd91c987cd4f7b"} Aug 04 11:34:33.268952 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.268917 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk"] Aug 04 11:34:33.272494 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.272477 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.275190 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.275165 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Aug 04 11:34:33.275190 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.275184 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Aug 04 11:34:33.275373 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.275229 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-8f58pcu9nmsle\"" Aug 04 11:34:33.275373 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.275183 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Aug 04 11:34:33.275373 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.275308 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-784q5\"" Aug 04 11:34:33.275557 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.275538 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Aug 04 11:34:33.276137 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.276124 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Aug 04 11:34:33.291749 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.291729 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk"] Aug 04 11:34:33.394924 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.394894 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a278c772-e331-4f77-989e-ac533e463d21-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.394924 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.394925 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a278c772-e331-4f77-989e-ac533e463d21-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.395100 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.394987 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a278c772-e331-4f77-989e-ac533e463d21-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.395100 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.395050 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq6s4\" (UniqueName: \"kubernetes.io/projected/a278c772-e331-4f77-989e-ac533e463d21-kube-api-access-qq6s4\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.395100 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.395078 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a278c772-e331-4f77-989e-ac533e463d21-metrics-client-ca\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.395246 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.395103 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a278c772-e331-4f77-989e-ac533e463d21-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.395246 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.395129 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a278c772-e331-4f77-989e-ac533e463d21-secret-grpc-tls\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.395246 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.395166 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a278c772-e331-4f77-989e-ac533e463d21-secret-thanos-querier-tls\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.496489 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.496458 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a278c772-e331-4f77-989e-ac533e463d21-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.496630 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.496510 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq6s4\" (UniqueName: \"kubernetes.io/projected/a278c772-e331-4f77-989e-ac533e463d21-kube-api-access-qq6s4\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.496630 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.496544 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a278c772-e331-4f77-989e-ac533e463d21-metrics-client-ca\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.496630 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.496575 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a278c772-e331-4f77-989e-ac533e463d21-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.496630 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.496597 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a278c772-e331-4f77-989e-ac533e463d21-secret-grpc-tls\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.496630 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.496628 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a278c772-e331-4f77-989e-ac533e463d21-secret-thanos-querier-tls\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.496907 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.496667 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a278c772-e331-4f77-989e-ac533e463d21-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.496981 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.496951 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a278c772-e331-4f77-989e-ac533e463d21-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.497689 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.497609 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a278c772-e331-4f77-989e-ac533e463d21-metrics-client-ca\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.499578 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.499543 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a278c772-e331-4f77-989e-ac533e463d21-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.499669 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.499568 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a278c772-e331-4f77-989e-ac533e463d21-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.499751 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.499730 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a278c772-e331-4f77-989e-ac533e463d21-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.499809 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.499796 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a278c772-e331-4f77-989e-ac533e463d21-secret-thanos-querier-tls\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.499872 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.499805 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a278c772-e331-4f77-989e-ac533e463d21-secret-grpc-tls\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.500291 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.500270 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a278c772-e331-4f77-989e-ac533e463d21-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.505049 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.505025 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq6s4\" (UniqueName: \"kubernetes.io/projected/a278c772-e331-4f77-989e-ac533e463d21-kube-api-access-qq6s4\") pod \"thanos-querier-54d87cb5fc-k4tvk\" (UID: \"a278c772-e331-4f77-989e-ac533e463d21\") " pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.519142 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.519100 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pjxbb" Aug 04 11:34:33.555827 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.555793 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bfqvz" event={"ID":"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3","Type":"ContainerStarted","Data":"6794d253c3971db07440cd689c4f1108d159bf3f3d914ecee3d9caced6c66b98"} Aug 04 11:34:33.556155 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.555853 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bfqvz" event={"ID":"fb8c73dd-6a50-4085-99ad-9ed552b4f3f3","Type":"ContainerStarted","Data":"247ad742faa7678fc6eb57cbd19d09f9a89aa0b2e875c210967c5541a2d01aff"} Aug 04 11:34:33.574222 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.574180 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bfqvz" podStartSLOduration=2.521481701 podStartE2EDuration="3.574165512s" totalImagesPullingTime="991.409714ms" totalInitContainerRuntime="61.274097ms" isStatefulPod=true podCreationTimestamp="2026-08-04 11:34:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:33.573304016 +0000 UTC m=+85.818023559" watchObservedRunningTime="2026-08-04 11:34:33.574165512 +0000 UTC m=+85.818885097" Aug 04 11:34:33.582351 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.582329 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:33.715992 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:33.715962 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk"] Aug 04 11:34:33.720628 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:34:33.719880 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda278c772_e331_4f77_989e_ac533e463d21.slice/crio-81503ff5b89775c4f7277e40132276844b69923008730ecf256237df05a06d25 WatchSource:0}: Error finding container 81503ff5b89775c4f7277e40132276844b69923008730ecf256237df05a06d25: Status 404 returned error can't find the container with id 81503ff5b89775c4f7277e40132276844b69923008730ecf256237df05a06d25 Aug 04 11:34:34.561032 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.560985 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" event={"ID":"a278c772-e331-4f77-989e-ac533e463d21","Type":"ContainerStarted","Data":"81503ff5b89775c4f7277e40132276844b69923008730ecf256237df05a06d25"} Aug 04 11:34:34.591412 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.591382 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-567c6fcf9-2wgtk"] Aug 04 11:34:34.594516 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.594498 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:34.597249 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.597219 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-j9vzv\"" Aug 04 11:34:34.597389 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.597363 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Aug 04 11:34:34.598296 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.598276 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Aug 04 11:34:34.598601 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.598584 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Aug 04 11:34:34.598795 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.598782 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Aug 04 11:34:34.598989 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.598974 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2c7o6ea0plchr\"" Aug 04 11:34:34.607788 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.607767 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-567c6fcf9-2wgtk"] Aug 04 11:34:34.706046 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.706012 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/692fda94-e203-446b-97cb-4a43752ac992-secret-metrics-server-tls\") pod \"metrics-server-567c6fcf9-2wgtk\" (UID: \"692fda94-e203-446b-97cb-4a43752ac992\") " pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:34.706214 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.706106 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/692fda94-e203-446b-97cb-4a43752ac992-client-ca-bundle\") pod \"metrics-server-567c6fcf9-2wgtk\" (UID: \"692fda94-e203-446b-97cb-4a43752ac992\") " pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:34.706214 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.706181 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/692fda94-e203-446b-97cb-4a43752ac992-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-567c6fcf9-2wgtk\" (UID: \"692fda94-e203-446b-97cb-4a43752ac992\") " pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:34.706342 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.706239 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/692fda94-e203-446b-97cb-4a43752ac992-audit-log\") pod \"metrics-server-567c6fcf9-2wgtk\" (UID: \"692fda94-e203-446b-97cb-4a43752ac992\") " pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:34.706342 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.706265 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/692fda94-e203-446b-97cb-4a43752ac992-metrics-server-audit-profiles\") pod \"metrics-server-567c6fcf9-2wgtk\" (UID: \"692fda94-e203-446b-97cb-4a43752ac992\") " pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:34.706342 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.706293 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xz8d\" (UniqueName: \"kubernetes.io/projected/692fda94-e203-446b-97cb-4a43752ac992-kube-api-access-8xz8d\") pod \"metrics-server-567c6fcf9-2wgtk\" (UID: \"692fda94-e203-446b-97cb-4a43752ac992\") " pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:34.706493 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.706376 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/692fda94-e203-446b-97cb-4a43752ac992-secret-metrics-server-client-certs\") pod \"metrics-server-567c6fcf9-2wgtk\" (UID: \"692fda94-e203-446b-97cb-4a43752ac992\") " pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:34.807436 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.807404 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/692fda94-e203-446b-97cb-4a43752ac992-client-ca-bundle\") pod \"metrics-server-567c6fcf9-2wgtk\" (UID: \"692fda94-e203-446b-97cb-4a43752ac992\") " pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:34.807603 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.807468 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/692fda94-e203-446b-97cb-4a43752ac992-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-567c6fcf9-2wgtk\" (UID: \"692fda94-e203-446b-97cb-4a43752ac992\") " pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:34.807603 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.807492 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/692fda94-e203-446b-97cb-4a43752ac992-audit-log\") pod \"metrics-server-567c6fcf9-2wgtk\" (UID: \"692fda94-e203-446b-97cb-4a43752ac992\") " pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:34.807603 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.807509 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/692fda94-e203-446b-97cb-4a43752ac992-metrics-server-audit-profiles\") pod \"metrics-server-567c6fcf9-2wgtk\" (UID: \"692fda94-e203-446b-97cb-4a43752ac992\") " pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:34.807603 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.807525 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xz8d\" (UniqueName: \"kubernetes.io/projected/692fda94-e203-446b-97cb-4a43752ac992-kube-api-access-8xz8d\") pod \"metrics-server-567c6fcf9-2wgtk\" (UID: \"692fda94-e203-446b-97cb-4a43752ac992\") " pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:34.807603 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.807549 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/692fda94-e203-446b-97cb-4a43752ac992-secret-metrics-server-client-certs\") pod \"metrics-server-567c6fcf9-2wgtk\" (UID: \"692fda94-e203-446b-97cb-4a43752ac992\") " pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:34.807603 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.807575 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/692fda94-e203-446b-97cb-4a43752ac992-secret-metrics-server-tls\") pod \"metrics-server-567c6fcf9-2wgtk\" (UID: \"692fda94-e203-446b-97cb-4a43752ac992\") " pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:34.807989 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.807966 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/692fda94-e203-446b-97cb-4a43752ac992-audit-log\") pod \"metrics-server-567c6fcf9-2wgtk\" (UID: \"692fda94-e203-446b-97cb-4a43752ac992\") " pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:34.808438 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.808411 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/692fda94-e203-446b-97cb-4a43752ac992-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-567c6fcf9-2wgtk\" (UID: \"692fda94-e203-446b-97cb-4a43752ac992\") " pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:34.808575 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.808555 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/692fda94-e203-446b-97cb-4a43752ac992-metrics-server-audit-profiles\") pod \"metrics-server-567c6fcf9-2wgtk\" (UID: \"692fda94-e203-446b-97cb-4a43752ac992\") " pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:34.809882 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.809858 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/692fda94-e203-446b-97cb-4a43752ac992-secret-metrics-server-client-certs\") pod \"metrics-server-567c6fcf9-2wgtk\" (UID: \"692fda94-e203-446b-97cb-4a43752ac992\") " pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:34.809983 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.809929 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/692fda94-e203-446b-97cb-4a43752ac992-secret-metrics-server-tls\") pod \"metrics-server-567c6fcf9-2wgtk\" (UID: \"692fda94-e203-446b-97cb-4a43752ac992\") " pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:34.810044 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.810008 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/692fda94-e203-446b-97cb-4a43752ac992-client-ca-bundle\") pod \"metrics-server-567c6fcf9-2wgtk\" (UID: \"692fda94-e203-446b-97cb-4a43752ac992\") " pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:34.817883 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.817826 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xz8d\" (UniqueName: \"kubernetes.io/projected/692fda94-e203-446b-97cb-4a43752ac992-kube-api-access-8xz8d\") pod \"metrics-server-567c6fcf9-2wgtk\" (UID: \"692fda94-e203-446b-97cb-4a43752ac992\") " pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:34.905792 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.905680 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:34.950509 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.950475 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-kdbbp"] Aug 04 11:34:34.955040 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.954060 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-kdbbp" Aug 04 11:34:34.956947 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.956920 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Aug 04 11:34:34.957203 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.957188 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-hljw4\"" Aug 04 11:34:34.962572 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:34.962509 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-kdbbp"] Aug 04 11:34:35.009950 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:35.009917 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c7c99760-43cf-4240-b1bb-713b92892169-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-kdbbp\" (UID: \"c7c99760-43cf-4240-b1bb-713b92892169\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-kdbbp" Aug 04 11:34:35.010104 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:35.010041 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb272\" (UniqueName: \"kubernetes.io/projected/c7c99760-43cf-4240-b1bb-713b92892169-kube-api-access-qb272\") pod \"monitoring-plugin-6dffbbd67-kdbbp\" (UID: \"c7c99760-43cf-4240-b1bb-713b92892169\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-kdbbp" Aug 04 11:34:35.013941 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:35.013909 2570 patch_prober.go:28] interesting pod/image-registry-789fcb8b4c-mqrqd container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 04 11:34:35.014056 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:35.013977 2570 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" podUID="f2fef570-6051-45ae-9317-592c2708fa0f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 11:34:35.069265 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:35.069237 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-567c6fcf9-2wgtk"] Aug 04 11:34:35.071784 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:34:35.071756 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod692fda94_e203_446b_97cb_4a43752ac992.slice/crio-a3cf21821e6e2acd23ced37692e7a2ca15e8a59a2e0f8aa411e5c1b98e7291e8 WatchSource:0}: Error finding container a3cf21821e6e2acd23ced37692e7a2ca15e8a59a2e0f8aa411e5c1b98e7291e8: Status 404 returned error can't find the container with id a3cf21821e6e2acd23ced37692e7a2ca15e8a59a2e0f8aa411e5c1b98e7291e8 Aug 04 11:34:35.110744 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:35.110715 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c7c99760-43cf-4240-b1bb-713b92892169-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-kdbbp\" (UID: \"c7c99760-43cf-4240-b1bb-713b92892169\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-kdbbp" Aug 04 11:34:35.110873 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:35.110776 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb272\" (UniqueName: \"kubernetes.io/projected/c7c99760-43cf-4240-b1bb-713b92892169-kube-api-access-qb272\") pod \"monitoring-plugin-6dffbbd67-kdbbp\" (UID: \"c7c99760-43cf-4240-b1bb-713b92892169\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-kdbbp" Aug 04 11:34:35.110873 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:34:35.110819 2570 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Aug 04 11:34:35.110970 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:34:35.110875 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7c99760-43cf-4240-b1bb-713b92892169-monitoring-plugin-cert podName:c7c99760-43cf-4240-b1bb-713b92892169 nodeName:}" failed. No retries permitted until 2026-08-04 11:34:35.610858732 +0000 UTC m=+87.855578253 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/c7c99760-43cf-4240-b1bb-713b92892169-monitoring-plugin-cert") pod "monitoring-plugin-6dffbbd67-kdbbp" (UID: "c7c99760-43cf-4240-b1bb-713b92892169") : secret "monitoring-plugin-cert" not found Aug 04 11:34:35.119213 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:35.119193 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb272\" (UniqueName: \"kubernetes.io/projected/c7c99760-43cf-4240-b1bb-713b92892169-kube-api-access-qb272\") pod \"monitoring-plugin-6dffbbd67-kdbbp\" (UID: \"c7c99760-43cf-4240-b1bb-713b92892169\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-kdbbp" Aug 04 11:34:35.566000 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:35.565961 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" event={"ID":"692fda94-e203-446b-97cb-4a43752ac992","Type":"ContainerStarted","Data":"a3cf21821e6e2acd23ced37692e7a2ca15e8a59a2e0f8aa411e5c1b98e7291e8"} Aug 04 11:34:35.614535 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:35.614497 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c7c99760-43cf-4240-b1bb-713b92892169-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-kdbbp\" (UID: \"c7c99760-43cf-4240-b1bb-713b92892169\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-kdbbp" Aug 04 11:34:35.617182 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:35.617159 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c7c99760-43cf-4240-b1bb-713b92892169-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-kdbbp\" (UID: \"c7c99760-43cf-4240-b1bb-713b92892169\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-kdbbp" Aug 04 11:34:35.865097 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:35.865073 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-kdbbp" Aug 04 11:34:36.012731 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.012695 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-kdbbp"] Aug 04 11:34:36.015794 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:34:36.015756 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7c99760_43cf_4240_b1bb_713b92892169.slice/crio-976999f475ccad9de811f04755e1395259598f40f693172377490198c0613a1b WatchSource:0}: Error finding container 976999f475ccad9de811f04755e1395259598f40f693172377490198c0613a1b: Status 404 returned error can't find the container with id 976999f475ccad9de811f04755e1395259598f40f693172377490198c0613a1b Aug 04 11:34:36.450929 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.450835 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 11:34:36.460695 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.460662 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.465531 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.465509 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Aug 04 11:34:36.465719 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.465573 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Aug 04 11:34:36.466751 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.466658 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Aug 04 11:34:36.466963 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.466928 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Aug 04 11:34:36.467075 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.466660 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Aug 04 11:34:36.467293 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.466824 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Aug 04 11:34:36.468359 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.467512 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Aug 04 11:34:36.468359 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.467645 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Aug 04 11:34:36.468359 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.467818 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Aug 04 11:34:36.468359 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.468136 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Aug 04 11:34:36.468359 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.468233 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Aug 04 11:34:36.468359 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.468348 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Aug 04 11:34:36.468359 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.468360 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-k6vkr\"" Aug 04 11:34:36.468776 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.468471 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4vof0u30sqtgv\"" Aug 04 11:34:36.477560 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.477540 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Aug 04 11:34:36.479666 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.479649 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-789fcb8b4c-mqrqd" Aug 04 11:34:36.480171 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.480152 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Aug 04 11:34:36.481199 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.481175 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 11:34:36.526418 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.526388 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.526418 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.526420 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.526669 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.526474 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e1ca00da-a816-4b8a-9417-09a67c7c0568-config-out\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.526669 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.526509 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.526669 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.526561 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.526669 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.526635 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.526669 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.526662 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j44ft\" (UniqueName: \"kubernetes.io/projected/e1ca00da-a816-4b8a-9417-09a67c7c0568-kube-api-access-j44ft\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.526897 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.526733 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.526897 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.526761 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.526897 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.526810 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.526897 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.526838 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.527793 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.527748 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.527903 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.527817 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.527903 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.527858 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.527903 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.527885 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e1ca00da-a816-4b8a-9417-09a67c7c0568-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.528051 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.527909 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.528051 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.527934 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.528051 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.527961 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-config\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.528051 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.527985 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.528051 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.528021 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-web-config\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.572008 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.571935 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-kdbbp" event={"ID":"c7c99760-43cf-4240-b1bb-713b92892169","Type":"ContainerStarted","Data":"976999f475ccad9de811f04755e1395259598f40f693172377490198c0613a1b"} Aug 04 11:34:36.574695 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.574667 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" event={"ID":"a278c772-e331-4f77-989e-ac533e463d21","Type":"ContainerStarted","Data":"24a0fd8e5a34bfda0bbcb4a61089ddc4dd7ee919a786c9b3ed4a6e6739c2f1f0"} Aug 04 11:34:36.574803 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.574723 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" event={"ID":"a278c772-e331-4f77-989e-ac533e463d21","Type":"ContainerStarted","Data":"2f1ede46bba6a65297eded9ad1f9795a886292c24ccab4cd828873f7f358c4aa"} Aug 04 11:34:36.574803 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.574748 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" event={"ID":"a278c772-e331-4f77-989e-ac533e463d21","Type":"ContainerStarted","Data":"cbc17804d321c31074e2b5cf839d648f06d39c29b07c1936a4c3e438de6de31c"} Aug 04 11:34:36.629492 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.629438 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.629682 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.629506 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.629682 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.629576 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.629682 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.629608 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.629682 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.629634 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.629682 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.629661 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e1ca00da-a816-4b8a-9417-09a67c7c0568-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.629937 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.629689 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.629937 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.629715 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.629937 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.629746 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-config\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.630105 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.630079 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.630714 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.630690 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-web-config\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.630777 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.630743 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.630777 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.630771 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.630836 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.630803 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e1ca00da-a816-4b8a-9417-09a67c7c0568-config-out\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.630875 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.630856 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.630875 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.630864 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.630936 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.630883 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.630936 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.630915 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.630994 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.630938 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j44ft\" (UniqueName: \"kubernetes.io/projected/e1ca00da-a816-4b8a-9417-09a67c7c0568-kube-api-access-j44ft\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.631035 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.631018 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.631071 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.631058 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.632168 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.630639 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.632630 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.632606 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.632710 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.632690 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.632979 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.632953 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.633126 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.633105 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.633190 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.633120 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.633190 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.633152 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.633439 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.633414 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.633655 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.633631 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.633729 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.633688 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.634429 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.634387 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e1ca00da-a816-4b8a-9417-09a67c7c0568-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.635038 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.634993 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e1ca00da-a816-4b8a-9417-09a67c7c0568-config-out\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.635139 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.635073 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-config\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.635642 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.635580 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.635849 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.635817 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.636362 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.636322 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-web-config\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.636940 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.636888 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.637693 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.637479 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.641943 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.641917 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j44ft\" (UniqueName: \"kubernetes.io/projected/e1ca00da-a816-4b8a-9417-09a67c7c0568-kube-api-access-j44ft\") pod \"prometheus-k8s-0\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:36.779008 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:36.778972 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:37.190787 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:37.190569 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 11:34:37.194965 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:34:37.194931 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1ca00da_a816_4b8a_9417_09a67c7c0568.slice/crio-b4aa876e5acd339739876c3795568da7fdba5472a36ea814296fcebde9e6bde7 WatchSource:0}: Error finding container b4aa876e5acd339739876c3795568da7fdba5472a36ea814296fcebde9e6bde7: Status 404 returned error can't find the container with id b4aa876e5acd339739876c3795568da7fdba5472a36ea814296fcebde9e6bde7 Aug 04 11:34:37.582237 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:37.582202 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" event={"ID":"a278c772-e331-4f77-989e-ac533e463d21","Type":"ContainerStarted","Data":"0d4e530b49cbe3bb21586bca472473c637ed2536ce27ddda95afd9c8980cfffb"} Aug 04 11:34:37.582919 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:37.582268 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" event={"ID":"a278c772-e331-4f77-989e-ac533e463d21","Type":"ContainerStarted","Data":"821e587bb05d03c413d1fda2c5b326eecb4d436836845643dd4acc84baebc39d"} Aug 04 11:34:37.582919 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:37.582296 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" event={"ID":"a278c772-e331-4f77-989e-ac533e463d21","Type":"ContainerStarted","Data":"a734de9f5b3dd7d79b52b0a24b9dcca5755cb1e159a0bb427874ea6f133addf5"} Aug 04 11:34:37.582919 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:37.582484 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:37.583880 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:37.583854 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1ca00da-a816-4b8a-9417-09a67c7c0568","Type":"ContainerStarted","Data":"b4aa876e5acd339739876c3795568da7fdba5472a36ea814296fcebde9e6bde7"} Aug 04 11:34:37.585430 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:37.585403 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" event={"ID":"692fda94-e203-446b-97cb-4a43752ac992","Type":"ContainerStarted","Data":"c7d438587a7e122aca3e47f8ccdd739d8a50446a054e8ba8d310da6f72cb488f"} Aug 04 11:34:37.610549 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:37.609578 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" podStartSLOduration=1.476203876 podStartE2EDuration="4.609559562s" totalImagesPullingTime="3.133355686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:34:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:37.605763944 +0000 UTC m=+89.850483487" watchObservedRunningTime="2026-08-04 11:34:37.609559562 +0000 UTC m=+89.854279105" Aug 04 11:34:37.627012 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:37.626966 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" podStartSLOduration=1.6578400229999999 podStartE2EDuration="3.626953286s" totalImagesPullingTime="1.969113263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:34:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:37.626564606 +0000 UTC m=+89.871284150" watchObservedRunningTime="2026-08-04 11:34:37.626953286 +0000 UTC m=+89.871672827" Aug 04 11:34:38.591762 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:38.591564 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-kdbbp" event={"ID":"c7c99760-43cf-4240-b1bb-713b92892169","Type":"ContainerStarted","Data":"e8d30cc5b00d5da2e58207431271c9718cfb2cd8f42172c5d3f6659ceadc5171"} Aug 04 11:34:38.591762 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:38.591707 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-kdbbp" Aug 04 11:34:38.593243 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:38.593220 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1ca00da-a816-4b8a-9417-09a67c7c0568","Type":"ContainerStarted","Data":"0cf8396ff228933902eadcc27a560a007ed06551548bd03611e415e280ef8180"} Aug 04 11:34:38.597290 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:38.597270 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-kdbbp" Aug 04 11:34:38.607523 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:38.607481 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-kdbbp" podStartSLOduration=2.687963261 podStartE2EDuration="4.607466845s" totalImagesPullingTime="1.919503584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:34:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:38.606397354 +0000 UTC m=+90.851116897" watchObservedRunningTime="2026-08-04 11:34:38.607466845 +0000 UTC m=+90.852186377" Aug 04 11:34:39.598611 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:39.598571 2570 generic.go:356] "Generic (PLEG): container finished" podID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerID="0cf8396ff228933902eadcc27a560a007ed06551548bd03611e415e280ef8180" exitCode=0 Aug 04 11:34:39.598975 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:39.598653 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1ca00da-a816-4b8a-9417-09a67c7c0568","Type":"ContainerDied","Data":"0cf8396ff228933902eadcc27a560a007ed06551548bd03611e415e280ef8180"} Aug 04 11:34:42.614897 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:42.614856 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1ca00da-a816-4b8a-9417-09a67c7c0568","Type":"ContainerStarted","Data":"549ea1748add750ce62dd9b9e99c5d0b65e0b62eabbebf85940d9a3129851870"} Aug 04 11:34:42.615338 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:42.614977 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1ca00da-a816-4b8a-9417-09a67c7c0568","Type":"ContainerStarted","Data":"c325d862cf3e99ee22204ada8dda77daf84ca45c60340a30660d5224dd5c23a3"} Aug 04 11:34:42.615338 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:42.615027 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1ca00da-a816-4b8a-9417-09a67c7c0568","Type":"ContainerStarted","Data":"10ef9e98ed90604a0d3b14e517dba18511ade78ca2e631d8a15a647cbdb22adb"} Aug 04 11:34:42.615338 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:42.615069 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1ca00da-a816-4b8a-9417-09a67c7c0568","Type":"ContainerStarted","Data":"98537db583447f450b4d37e13cc6bb4657bd3ff4f088298e6ed1625657851185"} Aug 04 11:34:42.615338 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:42.615110 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1ca00da-a816-4b8a-9417-09a67c7c0568","Type":"ContainerStarted","Data":"46de54cedd822453f3518971453ea05dcf10dbe15cb26b35c4c531dc084da144"} Aug 04 11:34:42.615338 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:42.615152 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1ca00da-a816-4b8a-9417-09a67c7c0568","Type":"ContainerStarted","Data":"b959a2aff0ccfcd69cf37eb36953a7a276b398cc7067b58674ed162b5424b9b4"} Aug 04 11:34:42.643882 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:42.643131 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.800963708 podStartE2EDuration="6.643109824s" totalImagesPullingTime="4.823811024s" totalInitContainerRuntime="18.335092ms" isStatefulPod=false podCreationTimestamp="2026-08-04 11:34:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:34:42.64200342 +0000 UTC m=+94.886722999" watchObservedRunningTime="2026-08-04 11:34:42.643109824 +0000 UTC m=+94.887829367" Aug 04 11:34:43.598432 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:43.598404 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-54d87cb5fc-k4tvk" Aug 04 11:34:46.779915 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:46.779870 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:34:54.519716 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:54.519687 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-42v7s" Aug 04 11:34:54.906700 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:54.906631 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:34:54.906700 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:34:54.906684 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:35:14.911134 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:14.911106 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:35:14.914970 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:14.914940 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-567c6fcf9-2wgtk" Aug 04 11:35:36.779691 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:36.779650 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:36.802148 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:36.802052 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:36.828201 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:36.828174 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:54.759822 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:54.759786 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 11:35:54.760259 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:54.760127 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="prometheus" containerID="cri-o://b959a2aff0ccfcd69cf37eb36953a7a276b398cc7067b58674ed162b5424b9b4" gracePeriod=600 Aug 04 11:35:54.760259 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:54.760188 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="config-reloader" containerID="cri-o://46de54cedd822453f3518971453ea05dcf10dbe15cb26b35c4c531dc084da144" gracePeriod=600 Aug 04 11:35:54.760259 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:54.760189 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="thanos-sidecar" containerID="cri-o://98537db583447f450b4d37e13cc6bb4657bd3ff4f088298e6ed1625657851185" gracePeriod=600 Aug 04 11:35:54.760430 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:54.760199 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="kube-rbac-proxy-thanos" containerID="cri-o://549ea1748add750ce62dd9b9e99c5d0b65e0b62eabbebf85940d9a3129851870" gracePeriod=600 Aug 04 11:35:54.760430 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:54.760199 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="kube-rbac-proxy-web" containerID="cri-o://10ef9e98ed90604a0d3b14e517dba18511ade78ca2e631d8a15a647cbdb22adb" gracePeriod=600 Aug 04 11:35:54.760561 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:54.760140 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="kube-rbac-proxy" containerID="cri-o://c325d862cf3e99ee22204ada8dda77daf84ca45c60340a30660d5224dd5c23a3" gracePeriod=600 Aug 04 11:35:54.883901 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:54.883874 2570 generic.go:356] "Generic (PLEG): container finished" podID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerID="549ea1748add750ce62dd9b9e99c5d0b65e0b62eabbebf85940d9a3129851870" exitCode=0 Aug 04 11:35:54.883901 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:54.883899 2570 generic.go:356] "Generic (PLEG): container finished" podID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerID="c325d862cf3e99ee22204ada8dda77daf84ca45c60340a30660d5224dd5c23a3" exitCode=0 Aug 04 11:35:54.883901 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:54.883905 2570 generic.go:356] "Generic (PLEG): container finished" podID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerID="98537db583447f450b4d37e13cc6bb4657bd3ff4f088298e6ed1625657851185" exitCode=0 Aug 04 11:35:54.884114 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:54.883910 2570 generic.go:356] "Generic (PLEG): container finished" podID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerID="46de54cedd822453f3518971453ea05dcf10dbe15cb26b35c4c531dc084da144" exitCode=0 Aug 04 11:35:54.884114 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:54.883915 2570 generic.go:356] "Generic (PLEG): container finished" podID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerID="b959a2aff0ccfcd69cf37eb36953a7a276b398cc7067b58674ed162b5424b9b4" exitCode=0 Aug 04 11:35:54.884114 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:54.883944 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1ca00da-a816-4b8a-9417-09a67c7c0568","Type":"ContainerDied","Data":"549ea1748add750ce62dd9b9e99c5d0b65e0b62eabbebf85940d9a3129851870"} Aug 04 11:35:54.884114 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:54.884010 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1ca00da-a816-4b8a-9417-09a67c7c0568","Type":"ContainerDied","Data":"c325d862cf3e99ee22204ada8dda77daf84ca45c60340a30660d5224dd5c23a3"} Aug 04 11:35:54.884114 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:54.884038 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1ca00da-a816-4b8a-9417-09a67c7c0568","Type":"ContainerDied","Data":"98537db583447f450b4d37e13cc6bb4657bd3ff4f088298e6ed1625657851185"} Aug 04 11:35:54.884114 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:54.884066 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1ca00da-a816-4b8a-9417-09a67c7c0568","Type":"ContainerDied","Data":"46de54cedd822453f3518971453ea05dcf10dbe15cb26b35c4c531dc084da144"} Aug 04 11:35:54.884114 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:54.884088 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1ca00da-a816-4b8a-9417-09a67c7c0568","Type":"ContainerDied","Data":"b959a2aff0ccfcd69cf37eb36953a7a276b398cc7067b58674ed162b5424b9b4"} Aug 04 11:35:55.890965 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:55.890930 2570 generic.go:356] "Generic (PLEG): container finished" podID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerID="10ef9e98ed90604a0d3b14e517dba18511ade78ca2e631d8a15a647cbdb22adb" exitCode=0 Aug 04 11:35:55.891268 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:55.890965 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1ca00da-a816-4b8a-9417-09a67c7c0568","Type":"ContainerDied","Data":"10ef9e98ed90604a0d3b14e517dba18511ade78ca2e631d8a15a647cbdb22adb"} Aug 04 11:35:55.999811 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:55.999786 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:56.074381 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.074303 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j44ft\" (UniqueName: \"kubernetes.io/projected/e1ca00da-a816-4b8a-9417-09a67c7c0568-kube-api-access-j44ft\") pod \"e1ca00da-a816-4b8a-9417-09a67c7c0568\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " Aug 04 11:35:56.074381 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.074347 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-rulefiles-2\") pod \"e1ca00da-a816-4b8a-9417-09a67c7c0568\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " Aug 04 11:35:56.074629 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.074379 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-web-config\") pod \"e1ca00da-a816-4b8a-9417-09a67c7c0568\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " Aug 04 11:35:56.074629 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.074407 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e1ca00da-a816-4b8a-9417-09a67c7c0568-tls-assets\") pod \"e1ca00da-a816-4b8a-9417-09a67c7c0568\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " Aug 04 11:35:56.074629 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.074438 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-configmap-kubelet-serving-ca-bundle\") pod \"e1ca00da-a816-4b8a-9417-09a67c7c0568\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " Aug 04 11:35:56.074629 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.074490 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-rulefiles-0\") pod \"e1ca00da-a816-4b8a-9417-09a67c7c0568\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " Aug 04 11:35:56.074629 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.074526 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"e1ca00da-a816-4b8a-9417-09a67c7c0568\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " Aug 04 11:35:56.074629 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.074557 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e1ca00da-a816-4b8a-9417-09a67c7c0568-config-out\") pod \"e1ca00da-a816-4b8a-9417-09a67c7c0568\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " Aug 04 11:35:56.074629 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.074597 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-rulefiles-1\") pod \"e1ca00da-a816-4b8a-9417-09a67c7c0568\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " Aug 04 11:35:56.074629 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.074623 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-configmap-serving-certs-ca-bundle\") pod \"e1ca00da-a816-4b8a-9417-09a67c7c0568\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " Aug 04 11:35:56.075022 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.074653 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-prometheus-k8s-tls\") pod \"e1ca00da-a816-4b8a-9417-09a67c7c0568\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " Aug 04 11:35:56.075022 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.074683 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"e1ca00da-a816-4b8a-9417-09a67c7c0568\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " Aug 04 11:35:56.075022 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.074720 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-config\") pod \"e1ca00da-a816-4b8a-9417-09a67c7c0568\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " Aug 04 11:35:56.075022 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.074730 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "e1ca00da-a816-4b8a-9417-09a67c7c0568" (UID: "e1ca00da-a816-4b8a-9417-09a67c7c0568"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:35:56.075022 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.074753 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-db\") pod \"e1ca00da-a816-4b8a-9417-09a67c7c0568\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " Aug 04 11:35:56.075022 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.074787 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-trusted-ca-bundle\") pod \"e1ca00da-a816-4b8a-9417-09a67c7c0568\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " Aug 04 11:35:56.075022 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.074830 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-metrics-client-certs\") pod \"e1ca00da-a816-4b8a-9417-09a67c7c0568\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " Aug 04 11:35:56.075022 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.074859 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-grpc-tls\") pod \"e1ca00da-a816-4b8a-9417-09a67c7c0568\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " Aug 04 11:35:56.075022 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.074950 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-kube-rbac-proxy\") pod \"e1ca00da-a816-4b8a-9417-09a67c7c0568\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " Aug 04 11:35:56.076081 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.076046 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "e1ca00da-a816-4b8a-9417-09a67c7c0568" (UID: "e1ca00da-a816-4b8a-9417-09a67c7c0568"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:35:56.076206 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.076184 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "e1ca00da-a816-4b8a-9417-09a67c7c0568" (UID: "e1ca00da-a816-4b8a-9417-09a67c7c0568"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:35:56.076904 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.076395 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-configmap-metrics-client-ca\") pod \"e1ca00da-a816-4b8a-9417-09a67c7c0568\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " Aug 04 11:35:56.076904 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.076465 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-thanos-prometheus-http-client-file\") pod \"e1ca00da-a816-4b8a-9417-09a67c7c0568\" (UID: \"e1ca00da-a816-4b8a-9417-09a67c7c0568\") " Aug 04 11:35:56.076904 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.076491 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "e1ca00da-a816-4b8a-9417-09a67c7c0568" (UID: "e1ca00da-a816-4b8a-9417-09a67c7c0568"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:35:56.076904 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.076739 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "e1ca00da-a816-4b8a-9417-09a67c7c0568" (UID: "e1ca00da-a816-4b8a-9417-09a67c7c0568"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:35:56.076904 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.076742 2570 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:35:56.076904 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.076774 2570 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:35:56.076904 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.076791 2570 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:35:56.076904 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.076807 2570 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:35:56.076904 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.076866 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "e1ca00da-a816-4b8a-9417-09a67c7c0568" (UID: "e1ca00da-a816-4b8a-9417-09a67c7c0568"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:35:56.077442 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.077264 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1ca00da-a816-4b8a-9417-09a67c7c0568-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e1ca00da-a816-4b8a-9417-09a67c7c0568" (UID: "e1ca00da-a816-4b8a-9417-09a67c7c0568"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:35:56.077645 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.077619 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "e1ca00da-a816-4b8a-9417-09a67c7c0568" (UID: "e1ca00da-a816-4b8a-9417-09a67c7c0568"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:35:56.078480 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.078437 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "e1ca00da-a816-4b8a-9417-09a67c7c0568" (UID: "e1ca00da-a816-4b8a-9417-09a67c7c0568"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 11:35:56.078834 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.078809 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-config" (OuterVolumeSpecName: "config") pod "e1ca00da-a816-4b8a-9417-09a67c7c0568" (UID: "e1ca00da-a816-4b8a-9417-09a67c7c0568"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 11:35:56.079157 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.079120 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "e1ca00da-a816-4b8a-9417-09a67c7c0568" (UID: "e1ca00da-a816-4b8a-9417-09a67c7c0568"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 11:35:56.079366 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.079341 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "e1ca00da-a816-4b8a-9417-09a67c7c0568" (UID: "e1ca00da-a816-4b8a-9417-09a67c7c0568"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 11:35:56.079846 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.079813 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1ca00da-a816-4b8a-9417-09a67c7c0568-kube-api-access-j44ft" (OuterVolumeSpecName: "kube-api-access-j44ft") pod "e1ca00da-a816-4b8a-9417-09a67c7c0568" (UID: "e1ca00da-a816-4b8a-9417-09a67c7c0568"). InnerVolumeSpecName "kube-api-access-j44ft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:35:56.079846 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.079823 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "e1ca00da-a816-4b8a-9417-09a67c7c0568" (UID: "e1ca00da-a816-4b8a-9417-09a67c7c0568"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 11:35:56.080216 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.080184 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1ca00da-a816-4b8a-9417-09a67c7c0568-config-out" (OuterVolumeSpecName: "config-out") pod "e1ca00da-a816-4b8a-9417-09a67c7c0568" (UID: "e1ca00da-a816-4b8a-9417-09a67c7c0568"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 11:35:56.080216 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.080201 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "e1ca00da-a816-4b8a-9417-09a67c7c0568" (UID: "e1ca00da-a816-4b8a-9417-09a67c7c0568"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 11:35:56.080339 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.080248 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "e1ca00da-a816-4b8a-9417-09a67c7c0568" (UID: "e1ca00da-a816-4b8a-9417-09a67c7c0568"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 11:35:56.080339 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.080266 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "e1ca00da-a816-4b8a-9417-09a67c7c0568" (UID: "e1ca00da-a816-4b8a-9417-09a67c7c0568"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 11:35:56.080537 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.080505 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "e1ca00da-a816-4b8a-9417-09a67c7c0568" (UID: "e1ca00da-a816-4b8a-9417-09a67c7c0568"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 11:35:56.089804 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.089784 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-web-config" (OuterVolumeSpecName: "web-config") pod "e1ca00da-a816-4b8a-9417-09a67c7c0568" (UID: "e1ca00da-a816-4b8a-9417-09a67c7c0568"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 11:35:56.177556 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.177505 2570 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-config\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:35:56.177556 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.177545 2570 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-k8s-db\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:35:56.177556 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.177555 2570 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:35:56.177556 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.177564 2570 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-metrics-client-certs\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:35:56.177556 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.177573 2570 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-grpc-tls\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:35:56.177830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.177581 2570 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-kube-rbac-proxy\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:35:56.177830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.177589 2570 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-configmap-metrics-client-ca\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:35:56.177830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.177598 2570 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:35:56.177830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.177607 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j44ft\" (UniqueName: \"kubernetes.io/projected/e1ca00da-a816-4b8a-9417-09a67c7c0568-kube-api-access-j44ft\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:35:56.177830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.177615 2570 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-web-config\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:35:56.177830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.177622 2570 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e1ca00da-a816-4b8a-9417-09a67c7c0568-tls-assets\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:35:56.177830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.177630 2570 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1ca00da-a816-4b8a-9417-09a67c7c0568-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:35:56.177830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.177638 2570 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:35:56.177830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.177647 2570 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e1ca00da-a816-4b8a-9417-09a67c7c0568-config-out\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:35:56.177830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.177655 2570 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:35:56.177830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.177663 2570 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e1ca00da-a816-4b8a-9417-09a67c7c0568-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:35:56.897007 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.896971 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e1ca00da-a816-4b8a-9417-09a67c7c0568","Type":"ContainerDied","Data":"b4aa876e5acd339739876c3795568da7fdba5472a36ea814296fcebde9e6bde7"} Aug 04 11:35:56.897353 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.897012 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:56.897353 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.897069 2570 scope.go:117] "RemoveContainer" containerID="549ea1748add750ce62dd9b9e99c5d0b65e0b62eabbebf85940d9a3129851870" Aug 04 11:35:56.905198 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.905181 2570 scope.go:117] "RemoveContainer" containerID="c325d862cf3e99ee22204ada8dda77daf84ca45c60340a30660d5224dd5c23a3" Aug 04 11:35:56.912172 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.912152 2570 scope.go:117] "RemoveContainer" containerID="10ef9e98ed90604a0d3b14e517dba18511ade78ca2e631d8a15a647cbdb22adb" Aug 04 11:35:56.918677 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.918662 2570 scope.go:117] "RemoveContainer" containerID="98537db583447f450b4d37e13cc6bb4657bd3ff4f088298e6ed1625657851185" Aug 04 11:35:56.921374 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.921348 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 11:35:56.925891 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.925873 2570 scope.go:117] "RemoveContainer" containerID="46de54cedd822453f3518971453ea05dcf10dbe15cb26b35c4c531dc084da144" Aug 04 11:35:56.926401 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.926374 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 11:35:56.931978 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.931963 2570 scope.go:117] "RemoveContainer" containerID="b959a2aff0ccfcd69cf37eb36953a7a276b398cc7067b58674ed162b5424b9b4" Aug 04 11:35:56.939956 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.939936 2570 scope.go:117] "RemoveContainer" containerID="0cf8396ff228933902eadcc27a560a007ed06551548bd03611e415e280ef8180" Aug 04 11:35:56.953030 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.953010 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 11:35:56.953270 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.953258 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="kube-rbac-proxy-thanos" Aug 04 11:35:56.953321 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.953270 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="kube-rbac-proxy-thanos" Aug 04 11:35:56.953321 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.953281 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="init-config-reloader" Aug 04 11:35:56.953321 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.953287 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="init-config-reloader" Aug 04 11:35:56.953321 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.953295 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="prometheus" Aug 04 11:35:56.953321 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.953301 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="prometheus" Aug 04 11:35:56.953321 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.953315 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="kube-rbac-proxy" Aug 04 11:35:56.953321 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.953320 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="kube-rbac-proxy" Aug 04 11:35:56.953569 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.953327 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="config-reloader" Aug 04 11:35:56.953569 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.953332 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="config-reloader" Aug 04 11:35:56.953569 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.953338 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="thanos-sidecar" Aug 04 11:35:56.953569 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.953343 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="thanos-sidecar" Aug 04 11:35:56.953569 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.953351 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="kube-rbac-proxy-web" Aug 04 11:35:56.953569 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.953356 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="kube-rbac-proxy-web" Aug 04 11:35:56.953569 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.953396 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="prometheus" Aug 04 11:35:56.953569 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.953402 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="config-reloader" Aug 04 11:35:56.953569 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.953409 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="thanos-sidecar" Aug 04 11:35:56.953569 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.953415 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="kube-rbac-proxy-web" Aug 04 11:35:56.953569 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.953420 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="kube-rbac-proxy" Aug 04 11:35:56.953569 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.953427 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" containerName="kube-rbac-proxy-thanos" Aug 04 11:35:56.958247 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.958230 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:56.960983 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.960958 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Aug 04 11:35:56.963000 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.962981 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-k6vkr\"" Aug 04 11:35:56.964003 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.963980 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Aug 04 11:35:56.964083 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.964029 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Aug 04 11:35:56.964146 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.964119 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Aug 04 11:35:56.964658 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.964593 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4vof0u30sqtgv\"" Aug 04 11:35:56.965227 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.964799 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Aug 04 11:35:56.965227 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.964833 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Aug 04 11:35:56.965227 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.964901 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Aug 04 11:35:56.965227 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.964993 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Aug 04 11:35:56.965227 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.965039 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Aug 04 11:35:56.965227 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.965106 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Aug 04 11:35:56.965227 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.965123 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Aug 04 11:35:56.967209 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.967194 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Aug 04 11:35:56.969327 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.969311 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Aug 04 11:35:56.970897 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.970881 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Aug 04 11:35:56.980123 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.980103 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 11:35:56.983533 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.983510 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00c9278b-a606-4fd1-9f15-6d43483c0088-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:56.983617 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.983543 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00c9278b-a606-4fd1-9f15-6d43483c0088-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:56.983617 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.983576 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/00c9278b-a606-4fd1-9f15-6d43483c0088-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:56.983729 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.983627 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/00c9278b-a606-4fd1-9f15-6d43483c0088-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:56.983729 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.983652 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:56.983729 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.983680 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:56.983729 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.983696 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00c9278b-a606-4fd1-9f15-6d43483c0088-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:56.983729 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.983718 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-config\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:56.983975 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.983741 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghvnn\" (UniqueName: \"kubernetes.io/projected/00c9278b-a606-4fd1-9f15-6d43483c0088-kube-api-access-ghvnn\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:56.983975 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.983781 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:56.983975 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.983830 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/00c9278b-a606-4fd1-9f15-6d43483c0088-config-out\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:56.983975 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.983859 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:56.983975 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.983875 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/00c9278b-a606-4fd1-9f15-6d43483c0088-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:56.983975 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.983891 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-web-config\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:56.983975 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.983908 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:56.983975 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.983962 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:56.984362 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.983991 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/00c9278b-a606-4fd1-9f15-6d43483c0088-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:56.984362 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.984007 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/00c9278b-a606-4fd1-9f15-6d43483c0088-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:56.984362 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.984026 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00c9278b-a606-4fd1-9f15-6d43483c0088-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:56.984362 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:56.984050 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.085316 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.085230 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/00c9278b-a606-4fd1-9f15-6d43483c0088-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.085316 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.085263 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/00c9278b-a606-4fd1-9f15-6d43483c0088-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.085316 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.085280 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00c9278b-a606-4fd1-9f15-6d43483c0088-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.085316 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.085300 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.085316 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.085319 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00c9278b-a606-4fd1-9f15-6d43483c0088-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.085714 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.085344 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00c9278b-a606-4fd1-9f15-6d43483c0088-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.085714 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.085361 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/00c9278b-a606-4fd1-9f15-6d43483c0088-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.085714 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.085379 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/00c9278b-a606-4fd1-9f15-6d43483c0088-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.085714 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.085401 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.085714 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.085427 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.085714 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.085469 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00c9278b-a606-4fd1-9f15-6d43483c0088-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.086137 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.086111 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00c9278b-a606-4fd1-9f15-6d43483c0088-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.086281 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.086201 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00c9278b-a606-4fd1-9f15-6d43483c0088-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.086385 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.086232 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00c9278b-a606-4fd1-9f15-6d43483c0088-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.086962 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.086267 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00c9278b-a606-4fd1-9f15-6d43483c0088-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.086962 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.086535 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-config\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.086962 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.086580 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghvnn\" (UniqueName: \"kubernetes.io/projected/00c9278b-a606-4fd1-9f15-6d43483c0088-kube-api-access-ghvnn\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.086962 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.086608 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.086962 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.086654 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/00c9278b-a606-4fd1-9f15-6d43483c0088-config-out\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.086962 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.086669 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/00c9278b-a606-4fd1-9f15-6d43483c0088-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.086962 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.086716 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.086962 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.086752 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/00c9278b-a606-4fd1-9f15-6d43483c0088-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.086962 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.086778 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-web-config\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.086962 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.086822 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.086962 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.086831 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/00c9278b-a606-4fd1-9f15-6d43483c0088-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.086962 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.086863 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.087614 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.087071 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/00c9278b-a606-4fd1-9f15-6d43483c0088-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.088988 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.088412 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.088988 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.088471 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.088988 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.088895 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.089328 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.089305 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/00c9278b-a606-4fd1-9f15-6d43483c0088-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.089725 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.089612 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.089725 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.089675 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.090080 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.090057 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/00c9278b-a606-4fd1-9f15-6d43483c0088-config-out\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.090679 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.090657 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/00c9278b-a606-4fd1-9f15-6d43483c0088-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.090899 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.090878 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-web-config\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.091219 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.091204 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.091256 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.091217 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-config\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.091778 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.091762 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00c9278b-a606-4fd1-9f15-6d43483c0088-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.096763 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.096744 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghvnn\" (UniqueName: \"kubernetes.io/projected/00c9278b-a606-4fd1-9f15-6d43483c0088-kube-api-access-ghvnn\") pod \"prometheus-k8s-0\" (UID: \"00c9278b-a606-4fd1-9f15-6d43483c0088\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.268033 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.267996 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:35:57.408038 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.408011 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 11:35:57.411340 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:35:57.411314 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00c9278b_a606_4fd1_9f15_6d43483c0088.slice/crio-d666c847687c9ce5361863b6f1eacf53c2320b17fb97237c69b39a56e04b0737 WatchSource:0}: Error finding container d666c847687c9ce5361863b6f1eacf53c2320b17fb97237c69b39a56e04b0737: Status 404 returned error can't find the container with id d666c847687c9ce5361863b6f1eacf53c2320b17fb97237c69b39a56e04b0737 Aug 04 11:35:57.901651 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.901616 2570 generic.go:356] "Generic (PLEG): container finished" podID="00c9278b-a606-4fd1-9f15-6d43483c0088" containerID="7f4c80c1ffe3f8eac35ef3f22401cc7f71ecea07636a049570eccd1a2fc4af49" exitCode=0 Aug 04 11:35:57.902047 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.901698 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"00c9278b-a606-4fd1-9f15-6d43483c0088","Type":"ContainerDied","Data":"7f4c80c1ffe3f8eac35ef3f22401cc7f71ecea07636a049570eccd1a2fc4af49"} Aug 04 11:35:57.902047 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:57.901799 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"00c9278b-a606-4fd1-9f15-6d43483c0088","Type":"ContainerStarted","Data":"d666c847687c9ce5361863b6f1eacf53c2320b17fb97237c69b39a56e04b0737"} Aug 04 11:35:58.197112 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:58.197083 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1ca00da-a816-4b8a-9417-09a67c7c0568" path="/var/lib/kubelet/pods/e1ca00da-a816-4b8a-9417-09a67c7c0568/volumes" Aug 04 11:35:58.910957 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:58.910915 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"00c9278b-a606-4fd1-9f15-6d43483c0088","Type":"ContainerStarted","Data":"48cb6842a23d5ffd1aafbf91b816366e219fd96ee34270512fe0b0b8d0cfe415"} Aug 04 11:35:58.911359 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:58.911012 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"00c9278b-a606-4fd1-9f15-6d43483c0088","Type":"ContainerStarted","Data":"fc3aa218588ddec169bb37ac67c62107388c6b67d81ccbd7a47ea9a0278b8029"} Aug 04 11:35:58.911359 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:58.911039 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"00c9278b-a606-4fd1-9f15-6d43483c0088","Type":"ContainerStarted","Data":"3989af6532c76dcfcc49f7fb8af64541fc5a8c22f0526d094f910dfd73983b9c"} Aug 04 11:35:58.911359 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:58.911064 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"00c9278b-a606-4fd1-9f15-6d43483c0088","Type":"ContainerStarted","Data":"2a6ca1acde78987f2d840067107ed1f07e267fd7fa0a085d1e86e1d3e21fb887"} Aug 04 11:35:58.911359 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:58.911088 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"00c9278b-a606-4fd1-9f15-6d43483c0088","Type":"ContainerStarted","Data":"c88f8ec6416f6fa4595a06774660303a5e68799d8d3735e8689b0e1ed8d8bcea"} Aug 04 11:35:58.911359 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:58.911122 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"00c9278b-a606-4fd1-9f15-6d43483c0088","Type":"ContainerStarted","Data":"2e3abe07401d1348a560950d82487aae82fd15c81a51c0f2da471378b8f3ffb2"} Aug 04 11:35:58.939900 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:35:58.939847 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.919231772 podStartE2EDuration="2.939832385s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.600613ms" isStatefulPod=false podCreationTimestamp="2026-08-04 11:35:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:35:58.939519891 +0000 UTC m=+171.184239434" watchObservedRunningTime="2026-08-04 11:35:58.939832385 +0000 UTC m=+171.184551926" Aug 04 11:36:02.268872 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:36:02.268782 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:36:57.268686 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:36:57.268653 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:36:57.283419 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:36:57.283394 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:36:58.138752 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:36:58.138723 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 11:38:08.064981 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:38:08.064948 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 11:38:08.070511 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:38:08.070488 2570 kubelet.go:1648] "Image garbage collection succeeded" Aug 04 11:38:08.071088 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:38:08.071070 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 11:39:04.141384 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.141312 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-d5b557dbd-gqt96"] Aug 04 11:39:04.144442 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.144425 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-gqt96" Aug 04 11:39:04.158694 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.158665 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Aug 04 11:39:04.158793 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.158705 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Aug 04 11:39:04.159241 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.159221 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Aug 04 11:39:04.159535 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.159518 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tgvcd\"" Aug 04 11:39:04.159797 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.159780 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Aug 04 11:39:04.160007 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.159992 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Aug 04 11:39:04.166809 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.166784 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d5b557dbd-gqt96"] Aug 04 11:39:04.265201 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.265158 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c942\" (UniqueName: \"kubernetes.io/projected/19ef2037-15c8-4873-b1f7-7bcd7beef87a-kube-api-access-4c942\") pod \"lws-controller-manager-d5b557dbd-gqt96\" (UID: \"19ef2037-15c8-4873-b1f7-7bcd7beef87a\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-gqt96" Aug 04 11:39:04.265364 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.265218 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19ef2037-15c8-4873-b1f7-7bcd7beef87a-cert\") pod \"lws-controller-manager-d5b557dbd-gqt96\" (UID: \"19ef2037-15c8-4873-b1f7-7bcd7beef87a\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-gqt96" Aug 04 11:39:04.265364 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.265296 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/19ef2037-15c8-4873-b1f7-7bcd7beef87a-metrics-cert\") pod \"lws-controller-manager-d5b557dbd-gqt96\" (UID: \"19ef2037-15c8-4873-b1f7-7bcd7beef87a\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-gqt96" Aug 04 11:39:04.265364 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.265345 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/19ef2037-15c8-4873-b1f7-7bcd7beef87a-manager-config\") pod \"lws-controller-manager-d5b557dbd-gqt96\" (UID: \"19ef2037-15c8-4873-b1f7-7bcd7beef87a\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-gqt96" Aug 04 11:39:04.366301 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.366272 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c942\" (UniqueName: \"kubernetes.io/projected/19ef2037-15c8-4873-b1f7-7bcd7beef87a-kube-api-access-4c942\") pod \"lws-controller-manager-d5b557dbd-gqt96\" (UID: \"19ef2037-15c8-4873-b1f7-7bcd7beef87a\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-gqt96" Aug 04 11:39:04.366434 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.366313 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19ef2037-15c8-4873-b1f7-7bcd7beef87a-cert\") pod \"lws-controller-manager-d5b557dbd-gqt96\" (UID: \"19ef2037-15c8-4873-b1f7-7bcd7beef87a\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-gqt96" Aug 04 11:39:04.366434 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.366335 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/19ef2037-15c8-4873-b1f7-7bcd7beef87a-metrics-cert\") pod \"lws-controller-manager-d5b557dbd-gqt96\" (UID: \"19ef2037-15c8-4873-b1f7-7bcd7beef87a\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-gqt96" Aug 04 11:39:04.366434 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.366374 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/19ef2037-15c8-4873-b1f7-7bcd7beef87a-manager-config\") pod \"lws-controller-manager-d5b557dbd-gqt96\" (UID: \"19ef2037-15c8-4873-b1f7-7bcd7beef87a\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-gqt96" Aug 04 11:39:04.367083 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.367065 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/19ef2037-15c8-4873-b1f7-7bcd7beef87a-manager-config\") pod \"lws-controller-manager-d5b557dbd-gqt96\" (UID: \"19ef2037-15c8-4873-b1f7-7bcd7beef87a\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-gqt96" Aug 04 11:39:04.368657 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.368634 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19ef2037-15c8-4873-b1f7-7bcd7beef87a-cert\") pod \"lws-controller-manager-d5b557dbd-gqt96\" (UID: \"19ef2037-15c8-4873-b1f7-7bcd7beef87a\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-gqt96" Aug 04 11:39:04.368786 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.368770 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/19ef2037-15c8-4873-b1f7-7bcd7beef87a-metrics-cert\") pod \"lws-controller-manager-d5b557dbd-gqt96\" (UID: \"19ef2037-15c8-4873-b1f7-7bcd7beef87a\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-gqt96" Aug 04 11:39:04.392858 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.392806 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c942\" (UniqueName: \"kubernetes.io/projected/19ef2037-15c8-4873-b1f7-7bcd7beef87a-kube-api-access-4c942\") pod \"lws-controller-manager-d5b557dbd-gqt96\" (UID: \"19ef2037-15c8-4873-b1f7-7bcd7beef87a\") " pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-gqt96" Aug 04 11:39:04.453331 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.453308 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-gqt96" Aug 04 11:39:04.614287 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.614243 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d5b557dbd-gqt96"] Aug 04 11:39:04.616400 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:39:04.616375 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19ef2037_15c8_4873_b1f7_7bcd7beef87a.slice/crio-b40558943ccbb963488a7f0102541595bc964b1ae9afab8ae8c7a63243a7203a WatchSource:0}: Error finding container b40558943ccbb963488a7f0102541595bc964b1ae9afab8ae8c7a63243a7203a: Status 404 returned error can't find the container with id b40558943ccbb963488a7f0102541595bc964b1ae9afab8ae8c7a63243a7203a Aug 04 11:39:04.618168 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:04.618153 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 11:39:05.581903 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:05.581868 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-gqt96" event={"ID":"19ef2037-15c8-4873-b1f7-7bcd7beef87a","Type":"ContainerStarted","Data":"b40558943ccbb963488a7f0102541595bc964b1ae9afab8ae8c7a63243a7203a"} Aug 04 11:39:06.232173 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.232139 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8"] Aug 04 11:39:06.242021 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.241996 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" Aug 04 11:39:06.251037 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.251008 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zv97q\"" Aug 04 11:39:06.251156 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.251092 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Aug 04 11:39:06.251570 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.251548 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Aug 04 11:39:06.251677 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.251575 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Aug 04 11:39:06.253506 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.253484 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Aug 04 11:39:06.274776 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.274750 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8"] Aug 04 11:39:06.282708 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.282683 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6de73521-bfa3-4490-b98d-104a8cfbc93c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-nhds8\" (UID: \"6de73521-bfa3-4490-b98d-104a8cfbc93c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" Aug 04 11:39:06.282825 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.282735 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rr94\" (UniqueName: \"kubernetes.io/projected/6de73521-bfa3-4490-b98d-104a8cfbc93c-kube-api-access-2rr94\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-nhds8\" (UID: \"6de73521-bfa3-4490-b98d-104a8cfbc93c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" Aug 04 11:39:06.282825 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.282799 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6de73521-bfa3-4490-b98d-104a8cfbc93c-manifests\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-nhds8\" (UID: \"6de73521-bfa3-4490-b98d-104a8cfbc93c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" Aug 04 11:39:06.282912 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.282835 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6de73521-bfa3-4490-b98d-104a8cfbc93c-tmp\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-nhds8\" (UID: \"6de73521-bfa3-4490-b98d-104a8cfbc93c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" Aug 04 11:39:06.282912 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.282901 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6de73521-bfa3-4490-b98d-104a8cfbc93c-webhook-cert\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-nhds8\" (UID: \"6de73521-bfa3-4490-b98d-104a8cfbc93c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" Aug 04 11:39:06.383924 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.383887 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6de73521-bfa3-4490-b98d-104a8cfbc93c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-nhds8\" (UID: \"6de73521-bfa3-4490-b98d-104a8cfbc93c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" Aug 04 11:39:06.384092 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.383941 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rr94\" (UniqueName: \"kubernetes.io/projected/6de73521-bfa3-4490-b98d-104a8cfbc93c-kube-api-access-2rr94\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-nhds8\" (UID: \"6de73521-bfa3-4490-b98d-104a8cfbc93c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" Aug 04 11:39:06.384092 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.383973 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6de73521-bfa3-4490-b98d-104a8cfbc93c-manifests\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-nhds8\" (UID: \"6de73521-bfa3-4490-b98d-104a8cfbc93c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" Aug 04 11:39:06.384092 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.383988 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6de73521-bfa3-4490-b98d-104a8cfbc93c-tmp\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-nhds8\" (UID: \"6de73521-bfa3-4490-b98d-104a8cfbc93c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" Aug 04 11:39:06.384092 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.384019 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6de73521-bfa3-4490-b98d-104a8cfbc93c-webhook-cert\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-nhds8\" (UID: \"6de73521-bfa3-4490-b98d-104a8cfbc93c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" Aug 04 11:39:06.384458 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.384428 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6de73521-bfa3-4490-b98d-104a8cfbc93c-manifests\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-nhds8\" (UID: \"6de73521-bfa3-4490-b98d-104a8cfbc93c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" Aug 04 11:39:06.386096 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.386078 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6de73521-bfa3-4490-b98d-104a8cfbc93c-tmp\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-nhds8\" (UID: \"6de73521-bfa3-4490-b98d-104a8cfbc93c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" Aug 04 11:39:06.386483 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.386438 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6de73521-bfa3-4490-b98d-104a8cfbc93c-webhook-cert\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-nhds8\" (UID: \"6de73521-bfa3-4490-b98d-104a8cfbc93c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" Aug 04 11:39:06.386538 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.386463 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6de73521-bfa3-4490-b98d-104a8cfbc93c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-nhds8\" (UID: \"6de73521-bfa3-4490-b98d-104a8cfbc93c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" Aug 04 11:39:06.392513 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.392496 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rr94\" (UniqueName: \"kubernetes.io/projected/6de73521-bfa3-4490-b98d-104a8cfbc93c-kube-api-access-2rr94\") pod \"opendatahub-operator-controller-manager-59f8dd78b9-nhds8\" (UID: \"6de73521-bfa3-4490-b98d-104a8cfbc93c\") " pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" Aug 04 11:39:06.554004 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.553929 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" Aug 04 11:39:06.847572 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:06.847546 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8"] Aug 04 11:39:06.849019 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:39:06.848994 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6de73521_bfa3_4490_b98d_104a8cfbc93c.slice/crio-b745d2a98003dc35d11c67db7d07ea4184fe0cf69e02a7cf7b21c9ba5cd33ce1 WatchSource:0}: Error finding container b745d2a98003dc35d11c67db7d07ea4184fe0cf69e02a7cf7b21c9ba5cd33ce1: Status 404 returned error can't find the container with id b745d2a98003dc35d11c67db7d07ea4184fe0cf69e02a7cf7b21c9ba5cd33ce1 Aug 04 11:39:07.594703 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:07.594665 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-gqt96" event={"ID":"19ef2037-15c8-4873-b1f7-7bcd7beef87a","Type":"ContainerStarted","Data":"ef596d03f2b99e18767ab4e08a8e17673051de2e95087c261e2dc1272ccee00e"} Aug 04 11:39:07.594895 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:07.594817 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-gqt96" Aug 04 11:39:07.596439 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:07.596408 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" event={"ID":"6de73521-bfa3-4490-b98d-104a8cfbc93c","Type":"ContainerStarted","Data":"b745d2a98003dc35d11c67db7d07ea4184fe0cf69e02a7cf7b21c9ba5cd33ce1"} Aug 04 11:39:07.614412 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:07.614346 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-gqt96" podStartSLOduration=1.45798904 podStartE2EDuration="3.614327964s" totalImagesPullingTime="2.156338924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:39:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:39:07.612726944 +0000 UTC m=+359.857446486" watchObservedRunningTime="2026-08-04 11:39:07.614327964 +0000 UTC m=+359.859047508" Aug 04 11:39:10.610392 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:10.610351 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" event={"ID":"6de73521-bfa3-4490-b98d-104a8cfbc93c","Type":"ContainerStarted","Data":"7608b03c4716f8cc5dc7feee5643a3efd1c8c947eb8d203ce4c91ed41b853745"} Aug 04 11:39:11.616233 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:11.616200 2570 generic.go:356] "Generic (PLEG): container finished" podID="6de73521-bfa3-4490-b98d-104a8cfbc93c" containerID="7608b03c4716f8cc5dc7feee5643a3efd1c8c947eb8d203ce4c91ed41b853745" exitCode=0 Aug 04 11:39:11.616624 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:11.616243 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" event={"ID":"6de73521-bfa3-4490-b98d-104a8cfbc93c","Type":"ContainerDied","Data":"7608b03c4716f8cc5dc7feee5643a3efd1c8c947eb8d203ce4c91ed41b853745"} Aug 04 11:39:12.621745 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:12.621712 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" event={"ID":"6de73521-bfa3-4490-b98d-104a8cfbc93c","Type":"ContainerStarted","Data":"7e439039faab51dbd6c47dbd567b2909a0581f003bc51f0eaa61d25f1ebea424"} Aug 04 11:39:12.622186 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:12.621842 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" Aug 04 11:39:12.645813 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:12.645768 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" podStartSLOduration=2.45007379 podStartE2EDuration="6.645753019s" totalImagesPullingTime="3.116546851s" totalInitContainerRuntime="1.079132378s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:39:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:39:12.644424686 +0000 UTC m=+364.889144227" watchObservedRunningTime="2026-08-04 11:39:12.645753019 +0000 UTC m=+364.890472561" Aug 04 11:39:18.603204 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:18.603169 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-d5b557dbd-gqt96" Aug 04 11:39:23.628629 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:23.628600 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-59f8dd78b9-nhds8" Aug 04 11:39:44.427883 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.427847 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht"] Aug 04 11:39:44.430991 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.430976 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:44.434118 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.434089 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Aug 04 11:39:44.434118 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.434102 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Aug 04 11:39:44.434343 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.434094 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Aug 04 11:39:44.434343 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.434157 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-vjj78\"" Aug 04 11:39:44.434432 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.434400 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Aug 04 11:39:44.434432 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.434400 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Aug 04 11:39:44.434693 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.434680 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Aug 04 11:39:44.450308 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.450288 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht"] Aug 04 11:39:44.594517 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.594488 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0741d66e-850c-4c18-a29a-3834ad2fd527-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-kdhht\" (UID: \"0741d66e-850c-4c18-a29a-3834ad2fd527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:44.594517 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.594521 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0741d66e-850c-4c18-a29a-3834ad2fd527-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-kdhht\" (UID: \"0741d66e-850c-4c18-a29a-3834ad2fd527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:44.594699 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.594540 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0741d66e-850c-4c18-a29a-3834ad2fd527-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-kdhht\" (UID: \"0741d66e-850c-4c18-a29a-3834ad2fd527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:44.594699 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.594578 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0741d66e-850c-4c18-a29a-3834ad2fd527-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-kdhht\" (UID: \"0741d66e-850c-4c18-a29a-3834ad2fd527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:44.594699 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.594622 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28cmh\" (UniqueName: \"kubernetes.io/projected/0741d66e-850c-4c18-a29a-3834ad2fd527-kube-api-access-28cmh\") pod \"istiod-openshift-gateway-5677ccb49b-kdhht\" (UID: \"0741d66e-850c-4c18-a29a-3834ad2fd527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:44.594699 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.594666 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0741d66e-850c-4c18-a29a-3834ad2fd527-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-kdhht\" (UID: \"0741d66e-850c-4c18-a29a-3834ad2fd527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:44.594846 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.594721 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0741d66e-850c-4c18-a29a-3834ad2fd527-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-kdhht\" (UID: \"0741d66e-850c-4c18-a29a-3834ad2fd527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:44.695834 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.695769 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0741d66e-850c-4c18-a29a-3834ad2fd527-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-kdhht\" (UID: \"0741d66e-850c-4c18-a29a-3834ad2fd527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:44.695834 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.695802 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0741d66e-850c-4c18-a29a-3834ad2fd527-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-kdhht\" (UID: \"0741d66e-850c-4c18-a29a-3834ad2fd527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:44.695834 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.695821 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0741d66e-850c-4c18-a29a-3834ad2fd527-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-kdhht\" (UID: \"0741d66e-850c-4c18-a29a-3834ad2fd527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:44.696010 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.695931 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0741d66e-850c-4c18-a29a-3834ad2fd527-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-kdhht\" (UID: \"0741d66e-850c-4c18-a29a-3834ad2fd527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:44.696010 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.695995 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0741d66e-850c-4c18-a29a-3834ad2fd527-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-kdhht\" (UID: \"0741d66e-850c-4c18-a29a-3834ad2fd527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:44.696086 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.696031 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28cmh\" (UniqueName: \"kubernetes.io/projected/0741d66e-850c-4c18-a29a-3834ad2fd527-kube-api-access-28cmh\") pod \"istiod-openshift-gateway-5677ccb49b-kdhht\" (UID: \"0741d66e-850c-4c18-a29a-3834ad2fd527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:44.696086 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.696074 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0741d66e-850c-4c18-a29a-3834ad2fd527-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-kdhht\" (UID: \"0741d66e-850c-4c18-a29a-3834ad2fd527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:44.696489 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.696440 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0741d66e-850c-4c18-a29a-3834ad2fd527-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-kdhht\" (UID: \"0741d66e-850c-4c18-a29a-3834ad2fd527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:44.698223 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.698193 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0741d66e-850c-4c18-a29a-3834ad2fd527-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-kdhht\" (UID: \"0741d66e-850c-4c18-a29a-3834ad2fd527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:44.698223 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.698208 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0741d66e-850c-4c18-a29a-3834ad2fd527-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-kdhht\" (UID: \"0741d66e-850c-4c18-a29a-3834ad2fd527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:44.698386 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.698266 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0741d66e-850c-4c18-a29a-3834ad2fd527-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-kdhht\" (UID: \"0741d66e-850c-4c18-a29a-3834ad2fd527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:44.698386 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.698333 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0741d66e-850c-4c18-a29a-3834ad2fd527-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-kdhht\" (UID: \"0741d66e-850c-4c18-a29a-3834ad2fd527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:44.709696 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.709674 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0741d66e-850c-4c18-a29a-3834ad2fd527-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-kdhht\" (UID: \"0741d66e-850c-4c18-a29a-3834ad2fd527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:44.709948 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.709928 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28cmh\" (UniqueName: \"kubernetes.io/projected/0741d66e-850c-4c18-a29a-3834ad2fd527-kube-api-access-28cmh\") pod \"istiod-openshift-gateway-5677ccb49b-kdhht\" (UID: \"0741d66e-850c-4c18-a29a-3834ad2fd527\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:44.740314 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.740292 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:44.883575 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:44.883540 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht"] Aug 04 11:39:44.886220 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:39:44.886193 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0741d66e_850c_4c18_a29a_3834ad2fd527.slice/crio-69dcea72c34384da547b5b1e6f9c5f47bfd023a84ec0f837a73cf9a5a5cca354 WatchSource:0}: Error finding container 69dcea72c34384da547b5b1e6f9c5f47bfd023a84ec0f837a73cf9a5a5cca354: Status 404 returned error can't find the container with id 69dcea72c34384da547b5b1e6f9c5f47bfd023a84ec0f837a73cf9a5a5cca354 Aug 04 11:39:45.753882 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:45.753850 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" event={"ID":"0741d66e-850c-4c18-a29a-3834ad2fd527","Type":"ContainerStarted","Data":"69dcea72c34384da547b5b1e6f9c5f47bfd023a84ec0f837a73cf9a5a5cca354"} Aug 04 11:39:47.967949 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:47.967903 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Aug 04 11:39:47.968320 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:47.967984 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Aug 04 11:39:48.767755 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:48.767717 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" event={"ID":"0741d66e-850c-4c18-a29a-3834ad2fd527","Type":"ContainerStarted","Data":"1b227590b4cd7a6ed8b44ec016ef0c3947b3207ab35f35995fb174d663a28a03"} Aug 04 11:39:48.767971 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:48.767840 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:39:48.807815 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:48.807767 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" podStartSLOduration=1.730539618 podStartE2EDuration="4.80775376s" totalImagesPullingTime="3.077214142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:39:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:39:48.805486612 +0000 UTC m=+401.050206148" watchObservedRunningTime="2026-08-04 11:39:48.80775376 +0000 UTC m=+401.052473301" Aug 04 11:39:49.773671 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:39:49.773644 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht" Aug 04 11:41:09.290037 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:41:09.290004 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw"] Aug 04 11:41:09.293180 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:41:09.293160 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw" Aug 04 11:41:09.296415 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:41:09.296394 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Aug 04 11:41:09.296542 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:41:09.296397 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-t66lf\"" Aug 04 11:41:09.296542 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:41:09.296487 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Aug 04 11:41:09.306673 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:41:09.306653 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw"] Aug 04 11:41:09.381355 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:41:09.381323 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2snql\" (UniqueName: \"kubernetes.io/projected/b3f4918b-044c-4357-9415-f6ed71e7ed27-kube-api-access-2snql\") pod \"kuadrant-operator-controller-manager-68cf589589-ftpmw\" (UID: \"b3f4918b-044c-4357-9415-f6ed71e7ed27\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw" Aug 04 11:41:09.381525 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:41:09.381384 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3f4918b-044c-4357-9415-f6ed71e7ed27-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-ftpmw\" (UID: \"b3f4918b-044c-4357-9415-f6ed71e7ed27\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw" Aug 04 11:41:09.482415 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:41:09.482367 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3f4918b-044c-4357-9415-f6ed71e7ed27-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-ftpmw\" (UID: \"b3f4918b-044c-4357-9415-f6ed71e7ed27\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw" Aug 04 11:41:09.482542 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:41:09.482461 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2snql\" (UniqueName: \"kubernetes.io/projected/b3f4918b-044c-4357-9415-f6ed71e7ed27-kube-api-access-2snql\") pod \"kuadrant-operator-controller-manager-68cf589589-ftpmw\" (UID: \"b3f4918b-044c-4357-9415-f6ed71e7ed27\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw" Aug 04 11:41:09.482752 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:41:09.482734 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3f4918b-044c-4357-9415-f6ed71e7ed27-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-ftpmw\" (UID: \"b3f4918b-044c-4357-9415-f6ed71e7ed27\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw" Aug 04 11:41:09.498706 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:41:09.498674 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2snql\" (UniqueName: \"kubernetes.io/projected/b3f4918b-044c-4357-9415-f6ed71e7ed27-kube-api-access-2snql\") pod \"kuadrant-operator-controller-manager-68cf589589-ftpmw\" (UID: \"b3f4918b-044c-4357-9415-f6ed71e7ed27\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw" Aug 04 11:41:09.602698 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:41:09.602637 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw" Aug 04 11:41:09.739074 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:41:09.739052 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw"] Aug 04 11:41:09.741339 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:41:09.741309 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3f4918b_044c_4357_9415_f6ed71e7ed27.slice/crio-a0da252d68a7919012a7d52b7ea7e14751f6df461b20ed042785e24746e2d117 WatchSource:0}: Error finding container a0da252d68a7919012a7d52b7ea7e14751f6df461b20ed042785e24746e2d117: Status 404 returned error can't find the container with id a0da252d68a7919012a7d52b7ea7e14751f6df461b20ed042785e24746e2d117 Aug 04 11:41:10.093285 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:41:10.093254 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw" event={"ID":"b3f4918b-044c-4357-9415-f6ed71e7ed27","Type":"ContainerStarted","Data":"a0da252d68a7919012a7d52b7ea7e14751f6df461b20ed042785e24746e2d117"} Aug 04 11:41:15.124357 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:41:15.124319 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw" event={"ID":"b3f4918b-044c-4357-9415-f6ed71e7ed27","Type":"ContainerStarted","Data":"6a1d12322eea6c34d2cb413c6f6303c3f06d0eee567ee0753b03ee9918fae891"} Aug 04 11:41:15.124801 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:41:15.124433 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw" Aug 04 11:41:15.146801 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:41:15.146749 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw" podStartSLOduration=1.7848744079999999 podStartE2EDuration="6.146732825s" totalImagesPullingTime="4.361858417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:41:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:41:15.143945725 +0000 UTC m=+487.388665268" watchObservedRunningTime="2026-08-04 11:41:15.146732825 +0000 UTC m=+487.391452371" Aug 04 11:41:26.131131 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:41:26.131103 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw" Aug 04 11:42:06.172872 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:42:06.172788 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jnxg5"] Aug 04 11:42:06.175321 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:42:06.175299 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jnxg5" Aug 04 11:42:06.177880 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:42:06.177857 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Aug 04 11:42:06.178192 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:42:06.178174 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Aug 04 11:42:06.179118 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:42:06.179096 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-d7pgt\"" Aug 04 11:42:06.179206 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:42:06.179133 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 11:42:06.184781 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:42:06.184758 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jnxg5"] Aug 04 11:42:06.255110 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:42:06.255069 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/494f1c12-d6ed-4720-8e7d-d33e5a21fe28-data\") pod \"postgres-69fb4594cf-jnxg5\" (UID: \"494f1c12-d6ed-4720-8e7d-d33e5a21fe28\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jnxg5" Aug 04 11:42:06.255291 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:42:06.255156 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbrl7\" (UniqueName: \"kubernetes.io/projected/494f1c12-d6ed-4720-8e7d-d33e5a21fe28-kube-api-access-nbrl7\") pod \"postgres-69fb4594cf-jnxg5\" (UID: \"494f1c12-d6ed-4720-8e7d-d33e5a21fe28\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jnxg5" Aug 04 11:42:06.355896 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:42:06.355868 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/494f1c12-d6ed-4720-8e7d-d33e5a21fe28-data\") pod \"postgres-69fb4594cf-jnxg5\" (UID: \"494f1c12-d6ed-4720-8e7d-d33e5a21fe28\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jnxg5" Aug 04 11:42:06.356079 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:42:06.355941 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbrl7\" (UniqueName: \"kubernetes.io/projected/494f1c12-d6ed-4720-8e7d-d33e5a21fe28-kube-api-access-nbrl7\") pod \"postgres-69fb4594cf-jnxg5\" (UID: \"494f1c12-d6ed-4720-8e7d-d33e5a21fe28\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jnxg5" Aug 04 11:42:06.356267 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:42:06.356246 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/494f1c12-d6ed-4720-8e7d-d33e5a21fe28-data\") pod \"postgres-69fb4594cf-jnxg5\" (UID: \"494f1c12-d6ed-4720-8e7d-d33e5a21fe28\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jnxg5" Aug 04 11:42:06.363833 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:42:06.363806 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbrl7\" (UniqueName: \"kubernetes.io/projected/494f1c12-d6ed-4720-8e7d-d33e5a21fe28-kube-api-access-nbrl7\") pod \"postgres-69fb4594cf-jnxg5\" (UID: \"494f1c12-d6ed-4720-8e7d-d33e5a21fe28\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jnxg5" Aug 04 11:42:06.485535 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:42:06.485507 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jnxg5" Aug 04 11:42:06.611618 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:42:06.611595 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jnxg5"] Aug 04 11:42:06.614195 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:42:06.614167 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod494f1c12_d6ed_4720_8e7d_d33e5a21fe28.slice/crio-96c25d9c301daf4052e155d413c059b5b227afb739c32a2a12c38b8bd94eaafd WatchSource:0}: Error finding container 96c25d9c301daf4052e155d413c059b5b227afb739c32a2a12c38b8bd94eaafd: Status 404 returned error can't find the container with id 96c25d9c301daf4052e155d413c059b5b227afb739c32a2a12c38b8bd94eaafd Aug 04 11:42:07.351403 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:42:07.351318 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jnxg5" event={"ID":"494f1c12-d6ed-4720-8e7d-d33e5a21fe28","Type":"ContainerStarted","Data":"96c25d9c301daf4052e155d413c059b5b227afb739c32a2a12c38b8bd94eaafd"} Aug 04 11:42:11.253481 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:42:11.253437 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Aug 04 11:42:11.371134 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:42:11.371093 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jnxg5" event={"ID":"494f1c12-d6ed-4720-8e7d-d33e5a21fe28","Type":"ContainerStarted","Data":"e6fcb33e99e6e3772a11f260ab08317ad4b4ee6cc21a9d17362e8317283a74b6"} Aug 04 11:42:11.371297 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:42:11.371169 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jnxg5" Aug 04 11:42:11.390721 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:42:11.390673 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jnxg5" podStartSLOduration=0.756229474 podStartE2EDuration="5.390658827s" totalImagesPullingTime="4.634429353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:42:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:42:11.388359955 +0000 UTC m=+543.633079499" watchObservedRunningTime="2026-08-04 11:42:11.390658827 +0000 UTC m=+543.635378369" Aug 04 11:42:17.402638 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:42:17.402610 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jnxg5" Aug 04 11:43:08.091852 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:08.091825 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 11:43:08.094855 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:08.094837 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 11:43:53.699886 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.699854 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf"] Aug 04 11:43:53.703177 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.703159 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:43:53.706283 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.706263 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Aug 04 11:43:53.707429 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.707410 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Aug 04 11:43:53.707554 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.707528 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5k874\"" Aug 04 11:43:53.707554 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.707544 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Aug 04 11:43:53.713957 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.713933 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf"] Aug 04 11:43:53.734253 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.734226 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2e4de77-e5dd-4e29-9138-7d79bdeebae8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc8jf\" (UID: \"b2e4de77-e5dd-4e29-9138-7d79bdeebae8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:43:53.734345 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.734262 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2e4de77-e5dd-4e29-9138-7d79bdeebae8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc8jf\" (UID: \"b2e4de77-e5dd-4e29-9138-7d79bdeebae8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:43:53.734345 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.734288 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2kbt\" (UniqueName: \"kubernetes.io/projected/b2e4de77-e5dd-4e29-9138-7d79bdeebae8-kube-api-access-b2kbt\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc8jf\" (UID: \"b2e4de77-e5dd-4e29-9138-7d79bdeebae8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:43:53.734413 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.734356 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2e4de77-e5dd-4e29-9138-7d79bdeebae8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc8jf\" (UID: \"b2e4de77-e5dd-4e29-9138-7d79bdeebae8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:43:53.734467 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.734409 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2e4de77-e5dd-4e29-9138-7d79bdeebae8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc8jf\" (UID: \"b2e4de77-e5dd-4e29-9138-7d79bdeebae8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:43:53.734499 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.734475 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2e4de77-e5dd-4e29-9138-7d79bdeebae8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc8jf\" (UID: \"b2e4de77-e5dd-4e29-9138-7d79bdeebae8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:43:53.834832 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.834801 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2e4de77-e5dd-4e29-9138-7d79bdeebae8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc8jf\" (UID: \"b2e4de77-e5dd-4e29-9138-7d79bdeebae8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:43:53.834945 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.834842 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2e4de77-e5dd-4e29-9138-7d79bdeebae8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc8jf\" (UID: \"b2e4de77-e5dd-4e29-9138-7d79bdeebae8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:43:53.834945 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.834899 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2e4de77-e5dd-4e29-9138-7d79bdeebae8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc8jf\" (UID: \"b2e4de77-e5dd-4e29-9138-7d79bdeebae8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:43:53.835024 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.834954 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2e4de77-e5dd-4e29-9138-7d79bdeebae8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc8jf\" (UID: \"b2e4de77-e5dd-4e29-9138-7d79bdeebae8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:43:53.835024 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.834993 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2kbt\" (UniqueName: \"kubernetes.io/projected/b2e4de77-e5dd-4e29-9138-7d79bdeebae8-kube-api-access-b2kbt\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc8jf\" (UID: \"b2e4de77-e5dd-4e29-9138-7d79bdeebae8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:43:53.835122 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.835036 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2e4de77-e5dd-4e29-9138-7d79bdeebae8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc8jf\" (UID: \"b2e4de77-e5dd-4e29-9138-7d79bdeebae8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:43:53.835429 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.835404 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2e4de77-e5dd-4e29-9138-7d79bdeebae8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc8jf\" (UID: \"b2e4de77-e5dd-4e29-9138-7d79bdeebae8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:43:53.835528 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.835415 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2e4de77-e5dd-4e29-9138-7d79bdeebae8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc8jf\" (UID: \"b2e4de77-e5dd-4e29-9138-7d79bdeebae8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:43:53.835528 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.835462 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2e4de77-e5dd-4e29-9138-7d79bdeebae8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc8jf\" (UID: \"b2e4de77-e5dd-4e29-9138-7d79bdeebae8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:43:53.837181 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.837162 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2e4de77-e5dd-4e29-9138-7d79bdeebae8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc8jf\" (UID: \"b2e4de77-e5dd-4e29-9138-7d79bdeebae8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:43:53.837432 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.837416 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2e4de77-e5dd-4e29-9138-7d79bdeebae8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc8jf\" (UID: \"b2e4de77-e5dd-4e29-9138-7d79bdeebae8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:43:53.844282 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:53.844259 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2kbt\" (UniqueName: \"kubernetes.io/projected/b2e4de77-e5dd-4e29-9138-7d79bdeebae8-kube-api-access-b2kbt\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc8jf\" (UID: \"b2e4de77-e5dd-4e29-9138-7d79bdeebae8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:43:54.013988 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:54.013957 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:43:54.144284 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:54.144261 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf"] Aug 04 11:43:54.147483 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:43:54.147432 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2e4de77_e5dd_4e29_9138_7d79bdeebae8.slice/crio-7cca062742a239fb7277118db4a280fc24fb65c7127797f75e948da41d9b36e5 WatchSource:0}: Error finding container 7cca062742a239fb7277118db4a280fc24fb65c7127797f75e948da41d9b36e5: Status 404 returned error can't find the container with id 7cca062742a239fb7277118db4a280fc24fb65c7127797f75e948da41d9b36e5 Aug 04 11:43:54.814643 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:54.814602 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" event={"ID":"b2e4de77-e5dd-4e29-9138-7d79bdeebae8","Type":"ContainerStarted","Data":"7cca062742a239fb7277118db4a280fc24fb65c7127797f75e948da41d9b36e5"} Aug 04 11:43:55.826058 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:55.826021 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" event={"ID":"b2e4de77-e5dd-4e29-9138-7d79bdeebae8","Type":"ContainerStarted","Data":"4b26fe55721c0a56621c88cc66dc134c8022c254d731a9f26948653b55c3dd78"} Aug 04 11:43:55.845865 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:43:55.845823 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" podStartSLOduration=1.878070584 podStartE2EDuration="2.845809264s" totalImagesPullingTime="967.73868ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:43:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:43:55.844263335 +0000 UTC m=+648.088982879" watchObservedRunningTime="2026-08-04 11:43:55.845809264 +0000 UTC m=+648.090528825" Aug 04 11:44:00.691843 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:00.691812 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv"] Aug 04 11:44:00.695243 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:00.695224 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:44:00.697637 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:00.697617 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Aug 04 11:44:00.705226 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:00.705201 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv"] Aug 04 11:44:00.780623 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:00.780597 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e26593c-4b41-45c9-964f-af9d699521ab-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv\" (UID: \"1e26593c-4b41-45c9-964f-af9d699521ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:44:00.780779 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:00.780634 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e26593c-4b41-45c9-964f-af9d699521ab-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv\" (UID: \"1e26593c-4b41-45c9-964f-af9d699521ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:44:00.780779 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:00.780709 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g47mg\" (UniqueName: \"kubernetes.io/projected/1e26593c-4b41-45c9-964f-af9d699521ab-kube-api-access-g47mg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv\" (UID: \"1e26593c-4b41-45c9-964f-af9d699521ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:44:00.780779 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:00.780749 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e26593c-4b41-45c9-964f-af9d699521ab-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv\" (UID: \"1e26593c-4b41-45c9-964f-af9d699521ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:44:00.780779 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:00.780766 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e26593c-4b41-45c9-964f-af9d699521ab-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv\" (UID: \"1e26593c-4b41-45c9-964f-af9d699521ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:44:00.780949 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:00.780789 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e26593c-4b41-45c9-964f-af9d699521ab-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv\" (UID: \"1e26593c-4b41-45c9-964f-af9d699521ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:44:00.881548 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:00.881520 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g47mg\" (UniqueName: \"kubernetes.io/projected/1e26593c-4b41-45c9-964f-af9d699521ab-kube-api-access-g47mg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv\" (UID: \"1e26593c-4b41-45c9-964f-af9d699521ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:44:00.881699 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:00.881563 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e26593c-4b41-45c9-964f-af9d699521ab-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv\" (UID: \"1e26593c-4b41-45c9-964f-af9d699521ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:44:00.881699 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:00.881588 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e26593c-4b41-45c9-964f-af9d699521ab-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv\" (UID: \"1e26593c-4b41-45c9-964f-af9d699521ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:44:00.881699 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:00.881610 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e26593c-4b41-45c9-964f-af9d699521ab-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv\" (UID: \"1e26593c-4b41-45c9-964f-af9d699521ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:44:00.881699 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:00.881642 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e26593c-4b41-45c9-964f-af9d699521ab-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv\" (UID: \"1e26593c-4b41-45c9-964f-af9d699521ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:44:00.881699 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:00.881663 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e26593c-4b41-45c9-964f-af9d699521ab-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv\" (UID: \"1e26593c-4b41-45c9-964f-af9d699521ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:44:00.881979 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:00.881956 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e26593c-4b41-45c9-964f-af9d699521ab-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv\" (UID: \"1e26593c-4b41-45c9-964f-af9d699521ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:44:00.882108 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:00.881999 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e26593c-4b41-45c9-964f-af9d699521ab-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv\" (UID: \"1e26593c-4b41-45c9-964f-af9d699521ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:44:00.882108 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:00.882051 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e26593c-4b41-45c9-964f-af9d699521ab-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv\" (UID: \"1e26593c-4b41-45c9-964f-af9d699521ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:44:00.883779 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:00.883763 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e26593c-4b41-45c9-964f-af9d699521ab-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv\" (UID: \"1e26593c-4b41-45c9-964f-af9d699521ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:44:00.884166 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:00.884147 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e26593c-4b41-45c9-964f-af9d699521ab-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv\" (UID: \"1e26593c-4b41-45c9-964f-af9d699521ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:44:00.890057 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:00.890035 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g47mg\" (UniqueName: \"kubernetes.io/projected/1e26593c-4b41-45c9-964f-af9d699521ab-kube-api-access-g47mg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv\" (UID: \"1e26593c-4b41-45c9-964f-af9d699521ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:44:01.006771 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:01.006744 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:44:01.154523 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:01.154498 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv"] Aug 04 11:44:01.156714 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:44:01.156687 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e26593c_4b41_45c9_964f_af9d699521ab.slice/crio-aaf6bc8bc8c8dd9795e33fb142fb2cbe4dd483804723370162e91d4c8c0ab557 WatchSource:0}: Error finding container aaf6bc8bc8c8dd9795e33fb142fb2cbe4dd483804723370162e91d4c8c0ab557: Status 404 returned error can't find the container with id aaf6bc8bc8c8dd9795e33fb142fb2cbe4dd483804723370162e91d4c8c0ab557 Aug 04 11:44:01.852824 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:01.852787 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" event={"ID":"1e26593c-4b41-45c9-964f-af9d699521ab","Type":"ContainerStarted","Data":"a15ffed03a0a6ba9d6b4665fd3dddbfba53b4ec6cc6dc415973d286e81fe5b98"} Aug 04 11:44:01.853335 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:01.852842 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" event={"ID":"1e26593c-4b41-45c9-964f-af9d699521ab","Type":"ContainerStarted","Data":"aaf6bc8bc8c8dd9795e33fb142fb2cbe4dd483804723370162e91d4c8c0ab557"} Aug 04 11:44:01.874065 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:01.874014 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" podStartSLOduration=1.617683585 podStartE2EDuration="1.873997592s" totalImagesPullingTime="256.314007ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:44:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:44:01.872246898 +0000 UTC m=+654.116966440" watchObservedRunningTime="2026-08-04 11:44:01.873997592 +0000 UTC m=+654.118717133" Aug 04 11:44:04.014339 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:04.014298 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:44:04.014800 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:04.014359 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:44:04.026594 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:04.026572 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:44:04.876861 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:04.876838 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc8jf" Aug 04 11:44:11.007126 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:11.007088 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:44:11.007500 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:11.007164 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:44:11.019251 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:11.019230 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:44:11.907226 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:44:11.907193 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv" Aug 04 11:48:08.121792 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:48:08.121712 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 11:48:08.123000 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:48:08.122970 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 11:53:08.149944 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:53:08.149914 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 11:53:08.151834 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:53:08.151815 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 11:54:25.291477 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:54:25.291429 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw"] Aug 04 11:54:25.293672 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:54:25.291635 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw" podUID="b3f4918b-044c-4357-9415-f6ed71e7ed27" containerName="manager" containerID="cri-o://6a1d12322eea6c34d2cb413c6f6303c3f06d0eee567ee0753b03ee9918fae891" gracePeriod=10 Aug 04 11:54:25.537078 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:54:25.537050 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw" Aug 04 11:54:25.577908 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:54:25.577793 2570 generic.go:356] "Generic (PLEG): container finished" podID="b3f4918b-044c-4357-9415-f6ed71e7ed27" containerID="6a1d12322eea6c34d2cb413c6f6303c3f06d0eee567ee0753b03ee9918fae891" exitCode=0 Aug 04 11:54:25.577908 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:54:25.577843 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw" Aug 04 11:54:25.577908 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:54:25.577858 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw" event={"ID":"b3f4918b-044c-4357-9415-f6ed71e7ed27","Type":"ContainerDied","Data":"6a1d12322eea6c34d2cb413c6f6303c3f06d0eee567ee0753b03ee9918fae891"} Aug 04 11:54:25.577908 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:54:25.577908 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw" event={"ID":"b3f4918b-044c-4357-9415-f6ed71e7ed27","Type":"ContainerDied","Data":"a0da252d68a7919012a7d52b7ea7e14751f6df461b20ed042785e24746e2d117"} Aug 04 11:54:25.578245 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:54:25.577934 2570 scope.go:117] "RemoveContainer" containerID="6a1d12322eea6c34d2cb413c6f6303c3f06d0eee567ee0753b03ee9918fae891" Aug 04 11:54:25.588789 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:54:25.588743 2570 scope.go:117] "RemoveContainer" containerID="6a1d12322eea6c34d2cb413c6f6303c3f06d0eee567ee0753b03ee9918fae891" Aug 04 11:54:25.589247 ip-10-0-128-47 kubenswrapper[2570]: E0804 11:54:25.589150 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a1d12322eea6c34d2cb413c6f6303c3f06d0eee567ee0753b03ee9918fae891\": container with ID starting with 6a1d12322eea6c34d2cb413c6f6303c3f06d0eee567ee0753b03ee9918fae891 not found: ID does not exist" containerID="6a1d12322eea6c34d2cb413c6f6303c3f06d0eee567ee0753b03ee9918fae891" Aug 04 11:54:25.589247 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:54:25.589190 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a1d12322eea6c34d2cb413c6f6303c3f06d0eee567ee0753b03ee9918fae891"} err="failed to get container status \"6a1d12322eea6c34d2cb413c6f6303c3f06d0eee567ee0753b03ee9918fae891\": rpc error: code = NotFound desc = could not find container \"6a1d12322eea6c34d2cb413c6f6303c3f06d0eee567ee0753b03ee9918fae891\": container with ID starting with 6a1d12322eea6c34d2cb413c6f6303c3f06d0eee567ee0753b03ee9918fae891 not found: ID does not exist" Aug 04 11:54:25.680669 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:54:25.680635 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3f4918b-044c-4357-9415-f6ed71e7ed27-extensions-socket-volume\") pod \"b3f4918b-044c-4357-9415-f6ed71e7ed27\" (UID: \"b3f4918b-044c-4357-9415-f6ed71e7ed27\") " Aug 04 11:54:25.680809 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:54:25.680722 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2snql\" (UniqueName: \"kubernetes.io/projected/b3f4918b-044c-4357-9415-f6ed71e7ed27-kube-api-access-2snql\") pod \"b3f4918b-044c-4357-9415-f6ed71e7ed27\" (UID: \"b3f4918b-044c-4357-9415-f6ed71e7ed27\") " Aug 04 11:54:25.681098 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:54:25.681077 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3f4918b-044c-4357-9415-f6ed71e7ed27-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b3f4918b-044c-4357-9415-f6ed71e7ed27" (UID: "b3f4918b-044c-4357-9415-f6ed71e7ed27"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 11:54:25.682906 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:54:25.682878 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3f4918b-044c-4357-9415-f6ed71e7ed27-kube-api-access-2snql" (OuterVolumeSpecName: "kube-api-access-2snql") pod "b3f4918b-044c-4357-9415-f6ed71e7ed27" (UID: "b3f4918b-044c-4357-9415-f6ed71e7ed27"). InnerVolumeSpecName "kube-api-access-2snql". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:54:25.781802 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:54:25.781771 2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3f4918b-044c-4357-9415-f6ed71e7ed27-extensions-socket-volume\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:54:25.781802 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:54:25.781797 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2snql\" (UniqueName: \"kubernetes.io/projected/b3f4918b-044c-4357-9415-f6ed71e7ed27-kube-api-access-2snql\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 11:54:25.902360 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:54:25.902329 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw"] Aug 04 11:54:25.909597 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:54:25.909573 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-ftpmw"] Aug 04 11:54:26.195807 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:54:26.195731 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3f4918b-044c-4357-9415-f6ed71e7ed27" path="/var/lib/kubelet/pods/b3f4918b-044c-4357-9415-f6ed71e7ed27/volumes" Aug 04 11:55:25.986574 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:55:25.986538 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd"] Aug 04 11:55:25.986973 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:55:25.986864 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3f4918b-044c-4357-9415-f6ed71e7ed27" containerName="manager" Aug 04 11:55:25.986973 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:55:25.986875 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3f4918b-044c-4357-9415-f6ed71e7ed27" containerName="manager" Aug 04 11:55:25.986973 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:55:25.986933 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3f4918b-044c-4357-9415-f6ed71e7ed27" containerName="manager" Aug 04 11:55:25.991013 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:55:25.990992 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd" Aug 04 11:55:25.995977 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:55:25.995958 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Aug 04 11:55:25.996075 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:55:25.996063 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-t66lf\"" Aug 04 11:55:25.996442 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:55:25.996426 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Aug 04 11:55:26.001969 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:55:26.001951 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4812319e-b1a4-477d-9db8-c9f8f2093458-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-xk9gd\" (UID: \"4812319e-b1a4-477d-9db8-c9f8f2093458\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd" Aug 04 11:55:26.002048 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:55:26.001985 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npfrp\" (UniqueName: \"kubernetes.io/projected/4812319e-b1a4-477d-9db8-c9f8f2093458-kube-api-access-npfrp\") pod \"kuadrant-operator-controller-manager-68cf589589-xk9gd\" (UID: \"4812319e-b1a4-477d-9db8-c9f8f2093458\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd" Aug 04 11:55:26.018404 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:55:26.018384 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd"] Aug 04 11:55:26.102891 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:55:26.102861 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4812319e-b1a4-477d-9db8-c9f8f2093458-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-xk9gd\" (UID: \"4812319e-b1a4-477d-9db8-c9f8f2093458\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd" Aug 04 11:55:26.103030 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:55:26.102906 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npfrp\" (UniqueName: \"kubernetes.io/projected/4812319e-b1a4-477d-9db8-c9f8f2093458-kube-api-access-npfrp\") pod \"kuadrant-operator-controller-manager-68cf589589-xk9gd\" (UID: \"4812319e-b1a4-477d-9db8-c9f8f2093458\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd" Aug 04 11:55:26.103220 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:55:26.103202 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4812319e-b1a4-477d-9db8-c9f8f2093458-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-xk9gd\" (UID: \"4812319e-b1a4-477d-9db8-c9f8f2093458\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd" Aug 04 11:55:26.134471 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:55:26.134432 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npfrp\" (UniqueName: \"kubernetes.io/projected/4812319e-b1a4-477d-9db8-c9f8f2093458-kube-api-access-npfrp\") pod \"kuadrant-operator-controller-manager-68cf589589-xk9gd\" (UID: \"4812319e-b1a4-477d-9db8-c9f8f2093458\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd" Aug 04 11:55:26.301952 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:55:26.301851 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd" Aug 04 11:55:26.431997 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:55:26.431967 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd"] Aug 04 11:55:26.434829 ip-10-0-128-47 kubenswrapper[2570]: W0804 11:55:26.434806 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4812319e_b1a4_477d_9db8_c9f8f2093458.slice/crio-a744022d478b05d18f1f5d862d7f61d597f9806b79e2b7e45d12a27c57cf8941 WatchSource:0}: Error finding container a744022d478b05d18f1f5d862d7f61d597f9806b79e2b7e45d12a27c57cf8941: Status 404 returned error can't find the container with id a744022d478b05d18f1f5d862d7f61d597f9806b79e2b7e45d12a27c57cf8941 Aug 04 11:55:26.852498 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:55:26.852465 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd" event={"ID":"4812319e-b1a4-477d-9db8-c9f8f2093458","Type":"ContainerStarted","Data":"5b5094b10add8144f28fd0f34aca207d5e8ecf8e37bef325e980cb690afa05b5"} Aug 04 11:55:26.852672 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:55:26.852518 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd" event={"ID":"4812319e-b1a4-477d-9db8-c9f8f2093458","Type":"ContainerStarted","Data":"a744022d478b05d18f1f5d862d7f61d597f9806b79e2b7e45d12a27c57cf8941"} Aug 04 11:55:26.852672 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:55:26.852555 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd" Aug 04 11:55:26.881830 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:55:26.881781 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd" podStartSLOduration=1.881765878 podStartE2EDuration="1.881765878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:55:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:55:26.879419596 +0000 UTC m=+1339.124139137" watchObservedRunningTime="2026-08-04 11:55:26.881765878 +0000 UTC m=+1339.126485419" Aug 04 11:55:37.860159 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:55:37.860118 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd" Aug 04 11:58:08.176140 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:58:08.176105 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 11:58:08.178247 ip-10-0-128-47 kubenswrapper[2570]: I0804 11:58:08.178225 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 12:03:08.203186 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:03:08.203157 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 12:03:08.205328 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:03:08.205308 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 12:03:44.578169 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:03:44.578133 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8"] Aug 04 12:03:44.581730 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:03:44.581713 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8" Aug 04 12:03:44.584183 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:03:44.584158 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-7db38b79-dockercfg-gk2n6\"" Aug 04 12:03:44.584286 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:03:44.584187 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-7db38b79\"" Aug 04 12:03:44.592181 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:03:44.592158 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8"] Aug 04 12:03:44.688594 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:03:44.688555 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fq9k\" (UniqueName: \"kubernetes.io/projected/c8e86a81-29ce-4e2c-b247-f87c9cd25364-kube-api-access-6fq9k\") pod \"payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8\" (UID: \"c8e86a81-29ce-4e2c-b247-f87c9cd25364\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8" Aug 04 12:03:44.688761 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:03:44.688598 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8e86a81-29ce-4e2c-b247-f87c9cd25364-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8\" (UID: \"c8e86a81-29ce-4e2c-b247-f87c9cd25364\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8" Aug 04 12:03:44.790077 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:03:44.790042 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fq9k\" (UniqueName: \"kubernetes.io/projected/c8e86a81-29ce-4e2c-b247-f87c9cd25364-kube-api-access-6fq9k\") pod \"payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8\" (UID: \"c8e86a81-29ce-4e2c-b247-f87c9cd25364\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8" Aug 04 12:03:44.790077 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:03:44.790076 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8e86a81-29ce-4e2c-b247-f87c9cd25364-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8\" (UID: \"c8e86a81-29ce-4e2c-b247-f87c9cd25364\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8" Aug 04 12:03:44.790798 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:03:44.790776 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8e86a81-29ce-4e2c-b247-f87c9cd25364-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8\" (UID: \"c8e86a81-29ce-4e2c-b247-f87c9cd25364\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8" Aug 04 12:03:44.801983 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:03:44.801957 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fq9k\" (UniqueName: \"kubernetes.io/projected/c8e86a81-29ce-4e2c-b247-f87c9cd25364-kube-api-access-6fq9k\") pod \"payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8\" (UID: \"c8e86a81-29ce-4e2c-b247-f87c9cd25364\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8" Aug 04 12:03:44.892351 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:03:44.892273 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8" Aug 04 12:03:45.031127 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:03:45.031099 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8"] Aug 04 12:03:45.033814 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:03:45.033783 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8e86a81_29ce_4e2c_b247_f87c9cd25364.slice/crio-0e96fa55e482bcea12799aabcc9af63c5a4d067d155e594dde47b773d9f8f3c5 WatchSource:0}: Error finding container 0e96fa55e482bcea12799aabcc9af63c5a4d067d155e594dde47b773d9f8f3c5: Status 404 returned error can't find the container with id 0e96fa55e482bcea12799aabcc9af63c5a4d067d155e594dde47b773d9f8f3c5 Aug 04 12:03:45.035805 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:03:45.035700 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 12:03:46.024670 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:03:46.024619 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8" event={"ID":"c8e86a81-29ce-4e2c-b247-f87c9cd25364","Type":"ContainerStarted","Data":"0e96fa55e482bcea12799aabcc9af63c5a4d067d155e594dde47b773d9f8f3c5"} Aug 04 12:03:48.035910 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:03:48.035866 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8" event={"ID":"c8e86a81-29ce-4e2c-b247-f87c9cd25364","Type":"ContainerStarted","Data":"419acfc024f91b15b3944e305b92db499fb3dea85d33932d78e0fcef89c37170"} Aug 04 12:03:48.036410 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:03:48.036024 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8" Aug 04 12:03:48.056492 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:03:48.056418 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8" podStartSLOduration=1.667274497 podStartE2EDuration="4.056403567s" totalImagesPullingTime="2.38912907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:03:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:03:48.055139347 +0000 UTC m=+1840.299858889" watchObservedRunningTime="2026-08-04 12:03:48.056403567 +0000 UTC m=+1840.301123108" Aug 04 12:03:59.044138 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:03:59.044102 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8" Aug 04 12:04:14.385247 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:14.385215 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8"] Aug 04 12:04:14.385679 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:14.385398 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8" podUID="c8e86a81-29ce-4e2c-b247-f87c9cd25364" containerName="payload-pre-processing" containerID="cri-o://419acfc024f91b15b3944e305b92db499fb3dea85d33932d78e0fcef89c37170" gracePeriod=30 Aug 04 12:04:14.628129 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:14.628104 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8" Aug 04 12:04:14.643763 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:14.643693 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fq9k\" (UniqueName: \"kubernetes.io/projected/c8e86a81-29ce-4e2c-b247-f87c9cd25364-kube-api-access-6fq9k\") pod \"c8e86a81-29ce-4e2c-b247-f87c9cd25364\" (UID: \"c8e86a81-29ce-4e2c-b247-f87c9cd25364\") " Aug 04 12:04:14.643886 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:14.643787 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8e86a81-29ce-4e2c-b247-f87c9cd25364-plugins-config-volume\") pod \"c8e86a81-29ce-4e2c-b247-f87c9cd25364\" (UID: \"c8e86a81-29ce-4e2c-b247-f87c9cd25364\") " Aug 04 12:04:14.644205 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:14.644162 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8e86a81-29ce-4e2c-b247-f87c9cd25364-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c8e86a81-29ce-4e2c-b247-f87c9cd25364" (UID: "c8e86a81-29ce-4e2c-b247-f87c9cd25364"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:04:14.648909 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:14.647719 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8e86a81-29ce-4e2c-b247-f87c9cd25364-kube-api-access-6fq9k" (OuterVolumeSpecName: "kube-api-access-6fq9k") pod "c8e86a81-29ce-4e2c-b247-f87c9cd25364" (UID: "c8e86a81-29ce-4e2c-b247-f87c9cd25364"). InnerVolumeSpecName "kube-api-access-6fq9k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:04:14.745226 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:14.745197 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8e86a81-29ce-4e2c-b247-f87c9cd25364-plugins-config-volume\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:04:14.745226 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:14.745224 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fq9k\" (UniqueName: \"kubernetes.io/projected/c8e86a81-29ce-4e2c-b247-f87c9cd25364-kube-api-access-6fq9k\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:04:15.161366 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:15.161327 2570 generic.go:356] "Generic (PLEG): container finished" podID="c8e86a81-29ce-4e2c-b247-f87c9cd25364" containerID="419acfc024f91b15b3944e305b92db499fb3dea85d33932d78e0fcef89c37170" exitCode=0 Aug 04 12:04:15.161588 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:15.161376 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8" Aug 04 12:04:15.161588 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:15.161411 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8" event={"ID":"c8e86a81-29ce-4e2c-b247-f87c9cd25364","Type":"ContainerDied","Data":"419acfc024f91b15b3944e305b92db499fb3dea85d33932d78e0fcef89c37170"} Aug 04 12:04:15.161588 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:15.161484 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8" event={"ID":"c8e86a81-29ce-4e2c-b247-f87c9cd25364","Type":"ContainerDied","Data":"0e96fa55e482bcea12799aabcc9af63c5a4d067d155e594dde47b773d9f8f3c5"} Aug 04 12:04:15.161588 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:15.161508 2570 scope.go:117] "RemoveContainer" containerID="419acfc024f91b15b3944e305b92db499fb3dea85d33932d78e0fcef89c37170" Aug 04 12:04:15.170499 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:15.170474 2570 scope.go:117] "RemoveContainer" containerID="419acfc024f91b15b3944e305b92db499fb3dea85d33932d78e0fcef89c37170" Aug 04 12:04:15.170773 ip-10-0-128-47 kubenswrapper[2570]: E0804 12:04:15.170752 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"419acfc024f91b15b3944e305b92db499fb3dea85d33932d78e0fcef89c37170\": container with ID starting with 419acfc024f91b15b3944e305b92db499fb3dea85d33932d78e0fcef89c37170 not found: ID does not exist" containerID="419acfc024f91b15b3944e305b92db499fb3dea85d33932d78e0fcef89c37170" Aug 04 12:04:15.170836 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:15.170780 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"419acfc024f91b15b3944e305b92db499fb3dea85d33932d78e0fcef89c37170"} err="failed to get container status \"419acfc024f91b15b3944e305b92db499fb3dea85d33932d78e0fcef89c37170\": rpc error: code = NotFound desc = could not find container \"419acfc024f91b15b3944e305b92db499fb3dea85d33932d78e0fcef89c37170\": container with ID starting with 419acfc024f91b15b3944e305b92db499fb3dea85d33932d78e0fcef89c37170 not found: ID does not exist" Aug 04 12:04:15.188975 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:15.188948 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8"] Aug 04 12:04:15.193962 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:15.193941 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8"] Aug 04 12:04:16.194206 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:16.194166 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8e86a81-29ce-4e2c-b247-f87c9cd25364" path="/var/lib/kubelet/pods/c8e86a81-29ce-4e2c-b247-f87c9cd25364/volumes" Aug 04 12:04:18.852601 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:18.852565 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657"] Aug 04 12:04:18.853163 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:18.853077 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8e86a81-29ce-4e2c-b247-f87c9cd25364" containerName="payload-pre-processing" Aug 04 12:04:18.853163 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:18.853096 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8e86a81-29ce-4e2c-b247-f87c9cd25364" containerName="payload-pre-processing" Aug 04 12:04:18.853277 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:18.853185 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8e86a81-29ce-4e2c-b247-f87c9cd25364" containerName="payload-pre-processing" Aug 04 12:04:18.857530 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:18.857509 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657" Aug 04 12:04:18.865335 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:18.865313 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-b6e21d41\"" Aug 04 12:04:18.865472 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:18.865404 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-b6e21d41-dockercfg-7dhwh\"" Aug 04 12:04:18.873070 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:18.873042 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657"] Aug 04 12:04:18.880307 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:18.880277 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlh6w\" (UniqueName: \"kubernetes.io/projected/56651c10-f0ac-4388-9846-712a7839cda9-kube-api-access-vlh6w\") pod \"payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657\" (UID: \"56651c10-f0ac-4388-9846-712a7839cda9\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657" Aug 04 12:04:18.880478 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:18.880384 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56651c10-f0ac-4388-9846-712a7839cda9-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657\" (UID: \"56651c10-f0ac-4388-9846-712a7839cda9\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657" Aug 04 12:04:18.981528 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:18.981490 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56651c10-f0ac-4388-9846-712a7839cda9-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657\" (UID: \"56651c10-f0ac-4388-9846-712a7839cda9\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657" Aug 04 12:04:18.981699 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:18.981573 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlh6w\" (UniqueName: \"kubernetes.io/projected/56651c10-f0ac-4388-9846-712a7839cda9-kube-api-access-vlh6w\") pod \"payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657\" (UID: \"56651c10-f0ac-4388-9846-712a7839cda9\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657" Aug 04 12:04:18.982133 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:18.982113 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56651c10-f0ac-4388-9846-712a7839cda9-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657\" (UID: \"56651c10-f0ac-4388-9846-712a7839cda9\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657" Aug 04 12:04:18.989550 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:18.989528 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlh6w\" (UniqueName: \"kubernetes.io/projected/56651c10-f0ac-4388-9846-712a7839cda9-kube-api-access-vlh6w\") pod \"payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657\" (UID: \"56651c10-f0ac-4388-9846-712a7839cda9\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657" Aug 04 12:04:19.167426 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:19.167343 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657" Aug 04 12:04:19.321567 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:19.321540 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657"] Aug 04 12:04:19.323523 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:04:19.323491 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56651c10_f0ac_4388_9846_712a7839cda9.slice/crio-5644f22bfb5f4712b3e05594a4adb1d8304cb02f9853e0e9794bca271b774ad4 WatchSource:0}: Error finding container 5644f22bfb5f4712b3e05594a4adb1d8304cb02f9853e0e9794bca271b774ad4: Status 404 returned error can't find the container with id 5644f22bfb5f4712b3e05594a4adb1d8304cb02f9853e0e9794bca271b774ad4 Aug 04 12:04:20.187545 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:20.187509 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657" event={"ID":"56651c10-f0ac-4388-9846-712a7839cda9","Type":"ContainerStarted","Data":"2524d59ba8329601c0003c392d5fe640f1e83a27f551bbaefc3633da9a8ff4e1"} Aug 04 12:04:20.188030 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:20.187573 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657" event={"ID":"56651c10-f0ac-4388-9846-712a7839cda9","Type":"ContainerStarted","Data":"5644f22bfb5f4712b3e05594a4adb1d8304cb02f9853e0e9794bca271b774ad4"} Aug 04 12:04:20.188030 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:20.187619 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657" Aug 04 12:04:20.206066 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:20.206024 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657" podStartSLOduration=2.206008253 podStartE2EDuration="2.206008253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:04:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:04:20.204358381 +0000 UTC m=+1872.449077948" watchObservedRunningTime="2026-08-04 12:04:20.206008253 +0000 UTC m=+1872.450727795" Aug 04 12:04:31.195360 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:31.195284 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657" Aug 04 12:04:48.872929 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:48.872888 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657"] Aug 04 12:04:48.876833 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:48.873629 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657" podUID="56651c10-f0ac-4388-9846-712a7839cda9" containerName="payload-pre-processing" containerID="cri-o://2524d59ba8329601c0003c392d5fe640f1e83a27f551bbaefc3633da9a8ff4e1" gracePeriod=30 Aug 04 12:04:49.125839 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:49.125771 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657" Aug 04 12:04:49.256081 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:49.256052 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56651c10-f0ac-4388-9846-712a7839cda9-plugins-config-volume\") pod \"56651c10-f0ac-4388-9846-712a7839cda9\" (UID: \"56651c10-f0ac-4388-9846-712a7839cda9\") " Aug 04 12:04:49.256232 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:49.256094 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlh6w\" (UniqueName: \"kubernetes.io/projected/56651c10-f0ac-4388-9846-712a7839cda9-kube-api-access-vlh6w\") pod \"56651c10-f0ac-4388-9846-712a7839cda9\" (UID: \"56651c10-f0ac-4388-9846-712a7839cda9\") " Aug 04 12:04:49.256441 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:49.256415 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56651c10-f0ac-4388-9846-712a7839cda9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "56651c10-f0ac-4388-9846-712a7839cda9" (UID: "56651c10-f0ac-4388-9846-712a7839cda9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:04:49.258095 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:49.258074 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56651c10-f0ac-4388-9846-712a7839cda9-kube-api-access-vlh6w" (OuterVolumeSpecName: "kube-api-access-vlh6w") pod "56651c10-f0ac-4388-9846-712a7839cda9" (UID: "56651c10-f0ac-4388-9846-712a7839cda9"). InnerVolumeSpecName "kube-api-access-vlh6w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:04:49.313063 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:49.313032 2570 generic.go:356] "Generic (PLEG): container finished" podID="56651c10-f0ac-4388-9846-712a7839cda9" containerID="2524d59ba8329601c0003c392d5fe640f1e83a27f551bbaefc3633da9a8ff4e1" exitCode=0 Aug 04 12:04:49.313218 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:49.313086 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657" Aug 04 12:04:49.313218 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:49.313130 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657" event={"ID":"56651c10-f0ac-4388-9846-712a7839cda9","Type":"ContainerDied","Data":"2524d59ba8329601c0003c392d5fe640f1e83a27f551bbaefc3633da9a8ff4e1"} Aug 04 12:04:49.313218 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:49.313181 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657" event={"ID":"56651c10-f0ac-4388-9846-712a7839cda9","Type":"ContainerDied","Data":"5644f22bfb5f4712b3e05594a4adb1d8304cb02f9853e0e9794bca271b774ad4"} Aug 04 12:04:49.313218 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:49.313209 2570 scope.go:117] "RemoveContainer" containerID="2524d59ba8329601c0003c392d5fe640f1e83a27f551bbaefc3633da9a8ff4e1" Aug 04 12:04:49.321559 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:49.321544 2570 scope.go:117] "RemoveContainer" containerID="2524d59ba8329601c0003c392d5fe640f1e83a27f551bbaefc3633da9a8ff4e1" Aug 04 12:04:49.321799 ip-10-0-128-47 kubenswrapper[2570]: E0804 12:04:49.321779 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2524d59ba8329601c0003c392d5fe640f1e83a27f551bbaefc3633da9a8ff4e1\": container with ID starting with 2524d59ba8329601c0003c392d5fe640f1e83a27f551bbaefc3633da9a8ff4e1 not found: ID does not exist" containerID="2524d59ba8329601c0003c392d5fe640f1e83a27f551bbaefc3633da9a8ff4e1" Aug 04 12:04:49.321857 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:49.321807 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2524d59ba8329601c0003c392d5fe640f1e83a27f551bbaefc3633da9a8ff4e1"} err="failed to get container status \"2524d59ba8329601c0003c392d5fe640f1e83a27f551bbaefc3633da9a8ff4e1\": rpc error: code = NotFound desc = could not find container \"2524d59ba8329601c0003c392d5fe640f1e83a27f551bbaefc3633da9a8ff4e1\": container with ID starting with 2524d59ba8329601c0003c392d5fe640f1e83a27f551bbaefc3633da9a8ff4e1 not found: ID does not exist" Aug 04 12:04:49.340376 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:49.340353 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657"] Aug 04 12:04:49.343101 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:49.343081 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657"] Aug 04 12:04:49.356809 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:49.356786 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56651c10-f0ac-4388-9846-712a7839cda9-plugins-config-volume\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:04:49.356919 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:49.356813 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlh6w\" (UniqueName: \"kubernetes.io/projected/56651c10-f0ac-4388-9846-712a7839cda9-kube-api-access-vlh6w\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:04:50.194117 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:50.194085 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56651c10-f0ac-4388-9846-712a7839cda9" path="/var/lib/kubelet/pods/56651c10-f0ac-4388-9846-712a7839cda9/volumes" Aug 04 12:04:54.164877 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:54.164845 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl"] Aug 04 12:04:54.165262 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:54.165177 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56651c10-f0ac-4388-9846-712a7839cda9" containerName="payload-pre-processing" Aug 04 12:04:54.165262 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:54.165188 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="56651c10-f0ac-4388-9846-712a7839cda9" containerName="payload-pre-processing" Aug 04 12:04:54.165262 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:54.165239 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="56651c10-f0ac-4388-9846-712a7839cda9" containerName="payload-pre-processing" Aug 04 12:04:54.169594 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:54.169580 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl" Aug 04 12:04:54.174219 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:54.174195 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-34f15601-dockercfg-vqqkb\"" Aug 04 12:04:54.174345 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:54.174221 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-34f15601\"" Aug 04 12:04:54.183625 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:54.183603 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl"] Aug 04 12:04:54.304413 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:54.304380 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbhdk\" (UniqueName: \"kubernetes.io/projected/6b499fe7-5ac1-4bf4-b163-c72518cb751c-kube-api-access-jbhdk\") pod \"payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl\" (UID: \"6b499fe7-5ac1-4bf4-b163-c72518cb751c\") " pod="openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl" Aug 04 12:04:54.304626 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:54.304564 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b499fe7-5ac1-4bf4-b163-c72518cb751c-plugins-config-volume\") pod \"payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl\" (UID: \"6b499fe7-5ac1-4bf4-b163-c72518cb751c\") " pod="openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl" Aug 04 12:04:54.406022 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:54.405984 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbhdk\" (UniqueName: \"kubernetes.io/projected/6b499fe7-5ac1-4bf4-b163-c72518cb751c-kube-api-access-jbhdk\") pod \"payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl\" (UID: \"6b499fe7-5ac1-4bf4-b163-c72518cb751c\") " pod="openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl" Aug 04 12:04:54.406220 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:54.406102 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b499fe7-5ac1-4bf4-b163-c72518cb751c-plugins-config-volume\") pod \"payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl\" (UID: \"6b499fe7-5ac1-4bf4-b163-c72518cb751c\") " pod="openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl" Aug 04 12:04:54.406839 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:54.406817 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b499fe7-5ac1-4bf4-b163-c72518cb751c-plugins-config-volume\") pod \"payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl\" (UID: \"6b499fe7-5ac1-4bf4-b163-c72518cb751c\") " pod="openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl" Aug 04 12:04:54.414371 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:54.414344 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbhdk\" (UniqueName: \"kubernetes.io/projected/6b499fe7-5ac1-4bf4-b163-c72518cb751c-kube-api-access-jbhdk\") pod \"payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl\" (UID: \"6b499fe7-5ac1-4bf4-b163-c72518cb751c\") " pod="openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl" Aug 04 12:04:54.479841 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:54.479811 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl" Aug 04 12:04:54.820697 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:54.820672 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl"] Aug 04 12:04:54.823230 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:04:54.823205 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b499fe7_5ac1_4bf4_b163_c72518cb751c.slice/crio-ec95ae87fc1207cfafedaf22fde1e42bcb1cc401e75b97da1479ab25ea9ee023 WatchSource:0}: Error finding container ec95ae87fc1207cfafedaf22fde1e42bcb1cc401e75b97da1479ab25ea9ee023: Status 404 returned error can't find the container with id ec95ae87fc1207cfafedaf22fde1e42bcb1cc401e75b97da1479ab25ea9ee023 Aug 04 12:04:55.341314 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:55.341280 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl" event={"ID":"6b499fe7-5ac1-4bf4-b163-c72518cb751c","Type":"ContainerStarted","Data":"e83c388c23660e7c55d339276df76692c2b33294dbe688b27d16180729c03bfc"} Aug 04 12:04:55.341776 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:55.341344 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl" event={"ID":"6b499fe7-5ac1-4bf4-b163-c72518cb751c","Type":"ContainerStarted","Data":"ec95ae87fc1207cfafedaf22fde1e42bcb1cc401e75b97da1479ab25ea9ee023"} Aug 04 12:04:55.341776 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:55.341385 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl" Aug 04 12:04:55.366112 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:04:55.366064 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl" podStartSLOduration=1.366048481 podStartE2EDuration="1.366048481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:04:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:04:55.363697479 +0000 UTC m=+1907.608417022" watchObservedRunningTime="2026-08-04 12:04:55.366048481 +0000 UTC m=+1907.610768023" Aug 04 12:05:06.348621 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:06.348592 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl" Aug 04 12:05:23.981822 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:23.981787 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl"] Aug 04 12:05:23.982407 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:23.982007 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl" podUID="6b499fe7-5ac1-4bf4-b163-c72518cb751c" containerName="payload-processing" containerID="cri-o://e83c388c23660e7c55d339276df76692c2b33294dbe688b27d16180729c03bfc" gracePeriod=30 Aug 04 12:05:24.232990 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:24.232933 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl" Aug 04 12:05:24.360570 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:24.360531 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbhdk\" (UniqueName: \"kubernetes.io/projected/6b499fe7-5ac1-4bf4-b163-c72518cb751c-kube-api-access-jbhdk\") pod \"6b499fe7-5ac1-4bf4-b163-c72518cb751c\" (UID: \"6b499fe7-5ac1-4bf4-b163-c72518cb751c\") " Aug 04 12:05:24.360762 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:24.360628 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b499fe7-5ac1-4bf4-b163-c72518cb751c-plugins-config-volume\") pod \"6b499fe7-5ac1-4bf4-b163-c72518cb751c\" (UID: \"6b499fe7-5ac1-4bf4-b163-c72518cb751c\") " Aug 04 12:05:24.361022 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:24.361001 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b499fe7-5ac1-4bf4-b163-c72518cb751c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6b499fe7-5ac1-4bf4-b163-c72518cb751c" (UID: "6b499fe7-5ac1-4bf4-b163-c72518cb751c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:05:24.362679 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:24.362655 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b499fe7-5ac1-4bf4-b163-c72518cb751c-kube-api-access-jbhdk" (OuterVolumeSpecName: "kube-api-access-jbhdk") pod "6b499fe7-5ac1-4bf4-b163-c72518cb751c" (UID: "6b499fe7-5ac1-4bf4-b163-c72518cb751c"). InnerVolumeSpecName "kube-api-access-jbhdk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:05:24.461578 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:24.461547 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b499fe7-5ac1-4bf4-b163-c72518cb751c-plugins-config-volume\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:05:24.461578 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:24.461571 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbhdk\" (UniqueName: \"kubernetes.io/projected/6b499fe7-5ac1-4bf4-b163-c72518cb751c-kube-api-access-jbhdk\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:05:24.464996 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:24.464967 2570 generic.go:356] "Generic (PLEG): container finished" podID="6b499fe7-5ac1-4bf4-b163-c72518cb751c" containerID="e83c388c23660e7c55d339276df76692c2b33294dbe688b27d16180729c03bfc" exitCode=0 Aug 04 12:05:24.465128 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:24.465014 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl" Aug 04 12:05:24.465128 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:24.465020 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl" event={"ID":"6b499fe7-5ac1-4bf4-b163-c72518cb751c","Type":"ContainerDied","Data":"e83c388c23660e7c55d339276df76692c2b33294dbe688b27d16180729c03bfc"} Aug 04 12:05:24.465128 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:24.465073 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl" event={"ID":"6b499fe7-5ac1-4bf4-b163-c72518cb751c","Type":"ContainerDied","Data":"ec95ae87fc1207cfafedaf22fde1e42bcb1cc401e75b97da1479ab25ea9ee023"} Aug 04 12:05:24.465128 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:24.465104 2570 scope.go:117] "RemoveContainer" containerID="e83c388c23660e7c55d339276df76692c2b33294dbe688b27d16180729c03bfc" Aug 04 12:05:24.473806 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:24.473787 2570 scope.go:117] "RemoveContainer" containerID="e83c388c23660e7c55d339276df76692c2b33294dbe688b27d16180729c03bfc" Aug 04 12:05:24.474060 ip-10-0-128-47 kubenswrapper[2570]: E0804 12:05:24.474039 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e83c388c23660e7c55d339276df76692c2b33294dbe688b27d16180729c03bfc\": container with ID starting with e83c388c23660e7c55d339276df76692c2b33294dbe688b27d16180729c03bfc not found: ID does not exist" containerID="e83c388c23660e7c55d339276df76692c2b33294dbe688b27d16180729c03bfc" Aug 04 12:05:24.474127 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:24.474069 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e83c388c23660e7c55d339276df76692c2b33294dbe688b27d16180729c03bfc"} err="failed to get container status \"e83c388c23660e7c55d339276df76692c2b33294dbe688b27d16180729c03bfc\": rpc error: code = NotFound desc = could not find container \"e83c388c23660e7c55d339276df76692c2b33294dbe688b27d16180729c03bfc\": container with ID starting with e83c388c23660e7c55d339276df76692c2b33294dbe688b27d16180729c03bfc not found: ID does not exist" Aug 04 12:05:24.486243 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:24.486115 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl"] Aug 04 12:05:24.488301 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:24.488283 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl"] Aug 04 12:05:26.195216 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:26.195181 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b499fe7-5ac1-4bf4-b163-c72518cb751c" path="/var/lib/kubelet/pods/6b499fe7-5ac1-4bf4-b163-c72518cb751c/volumes" Aug 04 12:05:30.454827 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:30.454795 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k"] Aug 04 12:05:30.455166 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:30.455115 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b499fe7-5ac1-4bf4-b163-c72518cb751c" containerName="payload-processing" Aug 04 12:05:30.455166 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:30.455126 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b499fe7-5ac1-4bf4-b163-c72518cb751c" containerName="payload-processing" Aug 04 12:05:30.455249 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:30.455192 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b499fe7-5ac1-4bf4-b163-c72518cb751c" containerName="payload-processing" Aug 04 12:05:30.459417 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:30.459399 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k" Aug 04 12:05:30.462189 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:30.462173 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-b9abcfc3-dockercfg-7p8gv\"" Aug 04 12:05:30.462281 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:30.462192 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-b9abcfc3\"" Aug 04 12:05:30.466957 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:30.466932 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k"] Aug 04 12:05:30.513079 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:30.513044 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zkbd\" (UniqueName: \"kubernetes.io/projected/e7eae578-0101-44f4-a772-6c23ac3ec430-kube-api-access-8zkbd\") pod \"payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k\" (UID: \"e7eae578-0101-44f4-a772-6c23ac3ec430\") " pod="openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k" Aug 04 12:05:30.513249 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:30.513115 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7eae578-0101-44f4-a772-6c23ac3ec430-plugins-config-volume\") pod \"payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k\" (UID: \"e7eae578-0101-44f4-a772-6c23ac3ec430\") " pod="openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k" Aug 04 12:05:30.613861 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:30.613825 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7eae578-0101-44f4-a772-6c23ac3ec430-plugins-config-volume\") pod \"payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k\" (UID: \"e7eae578-0101-44f4-a772-6c23ac3ec430\") " pod="openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k" Aug 04 12:05:30.614044 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:30.613893 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zkbd\" (UniqueName: \"kubernetes.io/projected/e7eae578-0101-44f4-a772-6c23ac3ec430-kube-api-access-8zkbd\") pod \"payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k\" (UID: \"e7eae578-0101-44f4-a772-6c23ac3ec430\") " pod="openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k" Aug 04 12:05:30.614544 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:30.614525 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7eae578-0101-44f4-a772-6c23ac3ec430-plugins-config-volume\") pod \"payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k\" (UID: \"e7eae578-0101-44f4-a772-6c23ac3ec430\") " pod="openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k" Aug 04 12:05:30.633499 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:30.633467 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zkbd\" (UniqueName: \"kubernetes.io/projected/e7eae578-0101-44f4-a772-6c23ac3ec430-kube-api-access-8zkbd\") pod \"payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k\" (UID: \"e7eae578-0101-44f4-a772-6c23ac3ec430\") " pod="openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k" Aug 04 12:05:30.770889 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:30.770852 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k" Aug 04 12:05:30.913241 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:30.913218 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k"] Aug 04 12:05:30.915956 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:05:30.915926 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7eae578_0101_44f4_a772_6c23ac3ec430.slice/crio-02c5af8edfc8a83ca69254fab7c6d7db79c5deb311356a61e65bb4f85e1df553 WatchSource:0}: Error finding container 02c5af8edfc8a83ca69254fab7c6d7db79c5deb311356a61e65bb4f85e1df553: Status 404 returned error can't find the container with id 02c5af8edfc8a83ca69254fab7c6d7db79c5deb311356a61e65bb4f85e1df553 Aug 04 12:05:31.497374 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:31.497336 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k" event={"ID":"e7eae578-0101-44f4-a772-6c23ac3ec430","Type":"ContainerStarted","Data":"bb4c30b1ba66a7478d24031a1f5f0fdb311d85fde394b239a53c1b443cdd8555"} Aug 04 12:05:31.497922 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:31.497396 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k" event={"ID":"e7eae578-0101-44f4-a772-6c23ac3ec430","Type":"ContainerStarted","Data":"02c5af8edfc8a83ca69254fab7c6d7db79c5deb311356a61e65bb4f85e1df553"} Aug 04 12:05:31.497922 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:31.497423 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k" Aug 04 12:05:42.503997 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:42.503968 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k" Aug 04 12:05:42.525155 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:05:42.525109 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k" podStartSLOduration=12.525096403 podStartE2EDuration="12.525096403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:05:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:05:31.519468826 +0000 UTC m=+1943.764188370" watchObservedRunningTime="2026-08-04 12:05:42.525096403 +0000 UTC m=+1954.769816034" Aug 04 12:06:00.284229 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:06:00.284133 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k"] Aug 04 12:06:00.284817 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:06:00.284358 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k" podUID="e7eae578-0101-44f4-a772-6c23ac3ec430" containerName="payload-processing" containerID="cri-o://bb4c30b1ba66a7478d24031a1f5f0fdb311d85fde394b239a53c1b443cdd8555" gracePeriod=30 Aug 04 12:06:00.531809 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:06:00.531778 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k" Aug 04 12:06:00.572004 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:06:00.571925 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7eae578-0101-44f4-a772-6c23ac3ec430-plugins-config-volume\") pod \"e7eae578-0101-44f4-a772-6c23ac3ec430\" (UID: \"e7eae578-0101-44f4-a772-6c23ac3ec430\") " Aug 04 12:06:00.572004 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:06:00.571982 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zkbd\" (UniqueName: \"kubernetes.io/projected/e7eae578-0101-44f4-a772-6c23ac3ec430-kube-api-access-8zkbd\") pod \"e7eae578-0101-44f4-a772-6c23ac3ec430\" (UID: \"e7eae578-0101-44f4-a772-6c23ac3ec430\") " Aug 04 12:06:00.572395 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:06:00.572365 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7eae578-0101-44f4-a772-6c23ac3ec430-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e7eae578-0101-44f4-a772-6c23ac3ec430" (UID: "e7eae578-0101-44f4-a772-6c23ac3ec430"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:06:00.574048 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:06:00.574026 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7eae578-0101-44f4-a772-6c23ac3ec430-kube-api-access-8zkbd" (OuterVolumeSpecName: "kube-api-access-8zkbd") pod "e7eae578-0101-44f4-a772-6c23ac3ec430" (UID: "e7eae578-0101-44f4-a772-6c23ac3ec430"). InnerVolumeSpecName "kube-api-access-8zkbd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:06:00.636093 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:06:00.636062 2570 generic.go:356] "Generic (PLEG): container finished" podID="e7eae578-0101-44f4-a772-6c23ac3ec430" containerID="bb4c30b1ba66a7478d24031a1f5f0fdb311d85fde394b239a53c1b443cdd8555" exitCode=0 Aug 04 12:06:00.636244 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:06:00.636115 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k" event={"ID":"e7eae578-0101-44f4-a772-6c23ac3ec430","Type":"ContainerDied","Data":"bb4c30b1ba66a7478d24031a1f5f0fdb311d85fde394b239a53c1b443cdd8555"} Aug 04 12:06:00.636244 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:06:00.636118 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k" Aug 04 12:06:00.636244 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:06:00.636163 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k" event={"ID":"e7eae578-0101-44f4-a772-6c23ac3ec430","Type":"ContainerDied","Data":"02c5af8edfc8a83ca69254fab7c6d7db79c5deb311356a61e65bb4f85e1df553"} Aug 04 12:06:00.636244 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:06:00.636186 2570 scope.go:117] "RemoveContainer" containerID="bb4c30b1ba66a7478d24031a1f5f0fdb311d85fde394b239a53c1b443cdd8555" Aug 04 12:06:00.644400 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:06:00.644384 2570 scope.go:117] "RemoveContainer" containerID="bb4c30b1ba66a7478d24031a1f5f0fdb311d85fde394b239a53c1b443cdd8555" Aug 04 12:06:00.644719 ip-10-0-128-47 kubenswrapper[2570]: E0804 12:06:00.644689 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb4c30b1ba66a7478d24031a1f5f0fdb311d85fde394b239a53c1b443cdd8555\": container with ID starting with bb4c30b1ba66a7478d24031a1f5f0fdb311d85fde394b239a53c1b443cdd8555 not found: ID does not exist" containerID="bb4c30b1ba66a7478d24031a1f5f0fdb311d85fde394b239a53c1b443cdd8555" Aug 04 12:06:00.644787 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:06:00.644717 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb4c30b1ba66a7478d24031a1f5f0fdb311d85fde394b239a53c1b443cdd8555"} err="failed to get container status \"bb4c30b1ba66a7478d24031a1f5f0fdb311d85fde394b239a53c1b443cdd8555\": rpc error: code = NotFound desc = could not find container \"bb4c30b1ba66a7478d24031a1f5f0fdb311d85fde394b239a53c1b443cdd8555\": container with ID starting with bb4c30b1ba66a7478d24031a1f5f0fdb311d85fde394b239a53c1b443cdd8555 not found: ID does not exist" Aug 04 12:06:00.659701 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:06:00.659680 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k"] Aug 04 12:06:00.663673 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:06:00.663653 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k"] Aug 04 12:06:00.673306 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:06:00.673289 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7eae578-0101-44f4-a772-6c23ac3ec430-plugins-config-volume\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:06:00.673379 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:06:00.673308 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zkbd\" (UniqueName: \"kubernetes.io/projected/e7eae578-0101-44f4-a772-6c23ac3ec430-kube-api-access-8zkbd\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:06:02.194349 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:06:02.194318 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7eae578-0101-44f4-a772-6c23ac3ec430" path="/var/lib/kubelet/pods/e7eae578-0101-44f4-a772-6c23ac3ec430/volumes" Aug 04 12:08:05.086798 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:05.086718 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s"] Aug 04 12:08:05.087277 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:05.087092 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7eae578-0101-44f4-a772-6c23ac3ec430" containerName="payload-processing" Aug 04 12:08:05.087277 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:05.087107 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7eae578-0101-44f4-a772-6c23ac3ec430" containerName="payload-processing" Aug 04 12:08:05.087277 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:05.087171 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7eae578-0101-44f4-a772-6c23ac3ec430" containerName="payload-processing" Aug 04 12:08:05.090195 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:05.090179 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s" Aug 04 12:08:05.092760 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:05.092743 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-6958d802\"" Aug 04 12:08:05.093350 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:05.093335 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-6958d802-dockercfg-g8l8j\"" Aug 04 12:08:05.102713 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:05.102692 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s"] Aug 04 12:08:05.135714 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:05.135677 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1d46a2b-f11b-4e8c-a383-1d4e9770ea84-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s\" (UID: \"d1d46a2b-f11b-4e8c-a383-1d4e9770ea84\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s" Aug 04 12:08:05.135861 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:05.135724 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z7l8\" (UniqueName: \"kubernetes.io/projected/d1d46a2b-f11b-4e8c-a383-1d4e9770ea84-kube-api-access-6z7l8\") pod \"payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s\" (UID: \"d1d46a2b-f11b-4e8c-a383-1d4e9770ea84\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s" Aug 04 12:08:05.236747 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:05.236691 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1d46a2b-f11b-4e8c-a383-1d4e9770ea84-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s\" (UID: \"d1d46a2b-f11b-4e8c-a383-1d4e9770ea84\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s" Aug 04 12:08:05.236938 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:05.236754 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z7l8\" (UniqueName: \"kubernetes.io/projected/d1d46a2b-f11b-4e8c-a383-1d4e9770ea84-kube-api-access-6z7l8\") pod \"payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s\" (UID: \"d1d46a2b-f11b-4e8c-a383-1d4e9770ea84\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s" Aug 04 12:08:05.237417 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:05.237398 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1d46a2b-f11b-4e8c-a383-1d4e9770ea84-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s\" (UID: \"d1d46a2b-f11b-4e8c-a383-1d4e9770ea84\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s" Aug 04 12:08:05.244832 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:05.244813 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z7l8\" (UniqueName: \"kubernetes.io/projected/d1d46a2b-f11b-4e8c-a383-1d4e9770ea84-kube-api-access-6z7l8\") pod \"payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s\" (UID: \"d1d46a2b-f11b-4e8c-a383-1d4e9770ea84\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s" Aug 04 12:08:05.400314 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:05.400234 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s" Aug 04 12:08:05.735622 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:05.735595 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s"] Aug 04 12:08:05.737896 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:08:05.737867 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1d46a2b_f11b_4e8c_a383_1d4e9770ea84.slice/crio-c18020c8b74664414c06170946755a2faa4f9863350b12b79739fd6be19ac1d2 WatchSource:0}: Error finding container c18020c8b74664414c06170946755a2faa4f9863350b12b79739fd6be19ac1d2: Status 404 returned error can't find the container with id c18020c8b74664414c06170946755a2faa4f9863350b12b79739fd6be19ac1d2 Aug 04 12:08:06.204090 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:06.204013 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s" event={"ID":"d1d46a2b-f11b-4e8c-a383-1d4e9770ea84","Type":"ContainerStarted","Data":"cb3f2f224fd83785762cb97126ff9c26d10b74a850f23422f30294e616c06eea"} Aug 04 12:08:06.204090 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:06.204061 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s" event={"ID":"d1d46a2b-f11b-4e8c-a383-1d4e9770ea84","Type":"ContainerStarted","Data":"c18020c8b74664414c06170946755a2faa4f9863350b12b79739fd6be19ac1d2"} Aug 04 12:08:06.204535 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:06.204142 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s" Aug 04 12:08:06.220478 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:06.220426 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s" podStartSLOduration=1.220413023 podStartE2EDuration="1.220413023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:08:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:08:06.21910716 +0000 UTC m=+2098.463826702" watchObservedRunningTime="2026-08-04 12:08:06.220413023 +0000 UTC m=+2098.465132616" Aug 04 12:08:08.228631 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:08.228603 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 12:08:08.232485 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:08.232439 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 12:08:17.212587 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:17.212542 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s" podUID="d1d46a2b-f11b-4e8c-a383-1d4e9770ea84" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.32:9004: connect: connection refused" Aug 04 12:08:27.212058 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:27.212027 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s" Aug 04 12:08:45.850505 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:45.850436 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s"] Aug 04 12:08:45.851090 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:45.850739 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s" podUID="d1d46a2b-f11b-4e8c-a383-1d4e9770ea84" containerName="payload-pre-processing" containerID="cri-o://cb3f2f224fd83785762cb97126ff9c26d10b74a850f23422f30294e616c06eea" gracePeriod=30 Aug 04 12:08:46.101726 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:46.101670 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s" Aug 04 12:08:46.199808 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:46.199765 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1d46a2b-f11b-4e8c-a383-1d4e9770ea84-plugins-config-volume\") pod \"d1d46a2b-f11b-4e8c-a383-1d4e9770ea84\" (UID: \"d1d46a2b-f11b-4e8c-a383-1d4e9770ea84\") " Aug 04 12:08:46.199973 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:46.199844 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6z7l8\" (UniqueName: \"kubernetes.io/projected/d1d46a2b-f11b-4e8c-a383-1d4e9770ea84-kube-api-access-6z7l8\") pod \"d1d46a2b-f11b-4e8c-a383-1d4e9770ea84\" (UID: \"d1d46a2b-f11b-4e8c-a383-1d4e9770ea84\") " Aug 04 12:08:46.200108 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:46.200084 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1d46a2b-f11b-4e8c-a383-1d4e9770ea84-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d1d46a2b-f11b-4e8c-a383-1d4e9770ea84" (UID: "d1d46a2b-f11b-4e8c-a383-1d4e9770ea84"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:08:46.201853 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:46.201828 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1d46a2b-f11b-4e8c-a383-1d4e9770ea84-kube-api-access-6z7l8" (OuterVolumeSpecName: "kube-api-access-6z7l8") pod "d1d46a2b-f11b-4e8c-a383-1d4e9770ea84" (UID: "d1d46a2b-f11b-4e8c-a383-1d4e9770ea84"). InnerVolumeSpecName "kube-api-access-6z7l8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:08:46.301403 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:46.301314 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1d46a2b-f11b-4e8c-a383-1d4e9770ea84-plugins-config-volume\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:08:46.301403 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:46.301347 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6z7l8\" (UniqueName: \"kubernetes.io/projected/d1d46a2b-f11b-4e8c-a383-1d4e9770ea84-kube-api-access-6z7l8\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:08:46.377112 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:46.377034 2570 generic.go:356] "Generic (PLEG): container finished" podID="d1d46a2b-f11b-4e8c-a383-1d4e9770ea84" containerID="cb3f2f224fd83785762cb97126ff9c26d10b74a850f23422f30294e616c06eea" exitCode=0 Aug 04 12:08:46.377112 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:46.377081 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s" Aug 04 12:08:46.377274 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:46.377117 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s" event={"ID":"d1d46a2b-f11b-4e8c-a383-1d4e9770ea84","Type":"ContainerDied","Data":"cb3f2f224fd83785762cb97126ff9c26d10b74a850f23422f30294e616c06eea"} Aug 04 12:08:46.377274 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:46.377174 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s" event={"ID":"d1d46a2b-f11b-4e8c-a383-1d4e9770ea84","Type":"ContainerDied","Data":"c18020c8b74664414c06170946755a2faa4f9863350b12b79739fd6be19ac1d2"} Aug 04 12:08:46.377274 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:46.377198 2570 scope.go:117] "RemoveContainer" containerID="cb3f2f224fd83785762cb97126ff9c26d10b74a850f23422f30294e616c06eea" Aug 04 12:08:46.388872 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:46.387761 2570 scope.go:117] "RemoveContainer" containerID="cb3f2f224fd83785762cb97126ff9c26d10b74a850f23422f30294e616c06eea" Aug 04 12:08:46.388872 ip-10-0-128-47 kubenswrapper[2570]: E0804 12:08:46.388080 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb3f2f224fd83785762cb97126ff9c26d10b74a850f23422f30294e616c06eea\": container with ID starting with cb3f2f224fd83785762cb97126ff9c26d10b74a850f23422f30294e616c06eea not found: ID does not exist" containerID="cb3f2f224fd83785762cb97126ff9c26d10b74a850f23422f30294e616c06eea" Aug 04 12:08:46.388872 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:46.388134 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb3f2f224fd83785762cb97126ff9c26d10b74a850f23422f30294e616c06eea"} err="failed to get container status \"cb3f2f224fd83785762cb97126ff9c26d10b74a850f23422f30294e616c06eea\": rpc error: code = NotFound desc = could not find container \"cb3f2f224fd83785762cb97126ff9c26d10b74a850f23422f30294e616c06eea\": container with ID starting with cb3f2f224fd83785762cb97126ff9c26d10b74a850f23422f30294e616c06eea not found: ID does not exist" Aug 04 12:08:46.403830 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:46.403794 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s"] Aug 04 12:08:46.405075 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:46.405056 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s"] Aug 04 12:08:48.194506 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:08:48.194475 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1d46a2b-f11b-4e8c-a383-1d4e9770ea84" path="/var/lib/kubelet/pods/d1d46a2b-f11b-4e8c-a383-1d4e9770ea84/volumes" Aug 04 12:09:33.566137 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:09:33.566103 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd"] Aug 04 12:09:33.566713 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:09:33.566298 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd" podUID="4812319e-b1a4-477d-9db8-c9f8f2093458" containerName="manager" containerID="cri-o://5b5094b10add8144f28fd0f34aca207d5e8ecf8e37bef325e980cb690afa05b5" gracePeriod=10 Aug 04 12:09:33.819281 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:09:33.819227 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd" Aug 04 12:09:33.915512 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:09:33.915485 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4812319e-b1a4-477d-9db8-c9f8f2093458-extensions-socket-volume\") pod \"4812319e-b1a4-477d-9db8-c9f8f2093458\" (UID: \"4812319e-b1a4-477d-9db8-c9f8f2093458\") " Aug 04 12:09:33.915690 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:09:33.915564 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npfrp\" (UniqueName: \"kubernetes.io/projected/4812319e-b1a4-477d-9db8-c9f8f2093458-kube-api-access-npfrp\") pod \"4812319e-b1a4-477d-9db8-c9f8f2093458\" (UID: \"4812319e-b1a4-477d-9db8-c9f8f2093458\") " Aug 04 12:09:33.915885 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:09:33.915861 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4812319e-b1a4-477d-9db8-c9f8f2093458-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4812319e-b1a4-477d-9db8-c9f8f2093458" (UID: "4812319e-b1a4-477d-9db8-c9f8f2093458"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 12:09:33.917602 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:09:33.917579 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4812319e-b1a4-477d-9db8-c9f8f2093458-kube-api-access-npfrp" (OuterVolumeSpecName: "kube-api-access-npfrp") pod "4812319e-b1a4-477d-9db8-c9f8f2093458" (UID: "4812319e-b1a4-477d-9db8-c9f8f2093458"). InnerVolumeSpecName "kube-api-access-npfrp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:09:34.016333 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:09:34.016299 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npfrp\" (UniqueName: \"kubernetes.io/projected/4812319e-b1a4-477d-9db8-c9f8f2093458-kube-api-access-npfrp\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:09:34.016333 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:09:34.016331 2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4812319e-b1a4-477d-9db8-c9f8f2093458-extensions-socket-volume\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:09:34.584389 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:09:34.584352 2570 generic.go:356] "Generic (PLEG): container finished" podID="4812319e-b1a4-477d-9db8-c9f8f2093458" containerID="5b5094b10add8144f28fd0f34aca207d5e8ecf8e37bef325e980cb690afa05b5" exitCode=0 Aug 04 12:09:34.584821 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:09:34.584392 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd" Aug 04 12:09:34.584821 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:09:34.584426 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd" event={"ID":"4812319e-b1a4-477d-9db8-c9f8f2093458","Type":"ContainerDied","Data":"5b5094b10add8144f28fd0f34aca207d5e8ecf8e37bef325e980cb690afa05b5"} Aug 04 12:09:34.584821 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:09:34.584495 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd" event={"ID":"4812319e-b1a4-477d-9db8-c9f8f2093458","Type":"ContainerDied","Data":"a744022d478b05d18f1f5d862d7f61d597f9806b79e2b7e45d12a27c57cf8941"} Aug 04 12:09:34.584821 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:09:34.584522 2570 scope.go:117] "RemoveContainer" containerID="5b5094b10add8144f28fd0f34aca207d5e8ecf8e37bef325e980cb690afa05b5" Aug 04 12:09:34.593847 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:09:34.593828 2570 scope.go:117] "RemoveContainer" containerID="5b5094b10add8144f28fd0f34aca207d5e8ecf8e37bef325e980cb690afa05b5" Aug 04 12:09:34.594137 ip-10-0-128-47 kubenswrapper[2570]: E0804 12:09:34.594117 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b5094b10add8144f28fd0f34aca207d5e8ecf8e37bef325e980cb690afa05b5\": container with ID starting with 5b5094b10add8144f28fd0f34aca207d5e8ecf8e37bef325e980cb690afa05b5 not found: ID does not exist" containerID="5b5094b10add8144f28fd0f34aca207d5e8ecf8e37bef325e980cb690afa05b5" Aug 04 12:09:34.594197 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:09:34.594144 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b5094b10add8144f28fd0f34aca207d5e8ecf8e37bef325e980cb690afa05b5"} err="failed to get container status \"5b5094b10add8144f28fd0f34aca207d5e8ecf8e37bef325e980cb690afa05b5\": rpc error: code = NotFound desc = could not find container \"5b5094b10add8144f28fd0f34aca207d5e8ecf8e37bef325e980cb690afa05b5\": container with ID starting with 5b5094b10add8144f28fd0f34aca207d5e8ecf8e37bef325e980cb690afa05b5 not found: ID does not exist" Aug 04 12:09:34.603169 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:09:34.603144 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd"] Aug 04 12:09:34.607025 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:09:34.607006 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xk9gd"] Aug 04 12:09:36.194433 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:09:36.194395 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4812319e-b1a4-477d-9db8-c9f8f2093458" path="/var/lib/kubelet/pods/4812319e-b1a4-477d-9db8-c9f8f2093458/volumes" Aug 04 12:10:49.226950 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.226865 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7skwx"] Aug 04 12:10:49.227426 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.227223 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4812319e-b1a4-477d-9db8-c9f8f2093458" containerName="manager" Aug 04 12:10:49.227426 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.227235 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4812319e-b1a4-477d-9db8-c9f8f2093458" containerName="manager" Aug 04 12:10:49.227426 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.227257 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1d46a2b-f11b-4e8c-a383-1d4e9770ea84" containerName="payload-pre-processing" Aug 04 12:10:49.227426 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.227263 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1d46a2b-f11b-4e8c-a383-1d4e9770ea84" containerName="payload-pre-processing" Aug 04 12:10:49.227426 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.227310 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1d46a2b-f11b-4e8c-a383-1d4e9770ea84" containerName="payload-pre-processing" Aug 04 12:10:49.227426 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.227319 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4812319e-b1a4-477d-9db8-c9f8f2093458" containerName="manager" Aug 04 12:10:49.230206 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.230191 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7skwx" Aug 04 12:10:49.232838 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.232815 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Aug 04 12:10:49.232838 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.232833 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Aug 04 12:10:49.233024 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.232852 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-t66lf\"" Aug 04 12:10:49.244588 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.244567 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7skwx"] Aug 04 12:10:49.349814 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.349763 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6453805-e250-4db3-9998-c2ae5888ee09-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-7skwx\" (UID: \"f6453805-e250-4db3-9998-c2ae5888ee09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7skwx" Aug 04 12:10:49.349814 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.349829 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv87v\" (UniqueName: \"kubernetes.io/projected/f6453805-e250-4db3-9998-c2ae5888ee09-kube-api-access-bv87v\") pod \"kuadrant-operator-controller-manager-68cf589589-7skwx\" (UID: \"f6453805-e250-4db3-9998-c2ae5888ee09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7skwx" Aug 04 12:10:49.450342 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.450297 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6453805-e250-4db3-9998-c2ae5888ee09-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-7skwx\" (UID: \"f6453805-e250-4db3-9998-c2ae5888ee09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7skwx" Aug 04 12:10:49.450550 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.450359 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv87v\" (UniqueName: \"kubernetes.io/projected/f6453805-e250-4db3-9998-c2ae5888ee09-kube-api-access-bv87v\") pod \"kuadrant-operator-controller-manager-68cf589589-7skwx\" (UID: \"f6453805-e250-4db3-9998-c2ae5888ee09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7skwx" Aug 04 12:10:49.450696 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.450679 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6453805-e250-4db3-9998-c2ae5888ee09-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-7skwx\" (UID: \"f6453805-e250-4db3-9998-c2ae5888ee09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7skwx" Aug 04 12:10:49.459814 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.459789 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv87v\" (UniqueName: \"kubernetes.io/projected/f6453805-e250-4db3-9998-c2ae5888ee09-kube-api-access-bv87v\") pod \"kuadrant-operator-controller-manager-68cf589589-7skwx\" (UID: \"f6453805-e250-4db3-9998-c2ae5888ee09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7skwx" Aug 04 12:10:49.540729 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.540638 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7skwx" Aug 04 12:10:49.676798 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.676775 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7skwx"] Aug 04 12:10:49.679590 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:10:49.679562 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6453805_e250_4db3_9998_c2ae5888ee09.slice/crio-3c3aeaff7d5058b8c242991ea31b78411e7e60425ae9507088483b465ae1bade WatchSource:0}: Error finding container 3c3aeaff7d5058b8c242991ea31b78411e7e60425ae9507088483b465ae1bade: Status 404 returned error can't find the container with id 3c3aeaff7d5058b8c242991ea31b78411e7e60425ae9507088483b465ae1bade Aug 04 12:10:49.910720 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.910639 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7skwx" event={"ID":"f6453805-e250-4db3-9998-c2ae5888ee09","Type":"ContainerStarted","Data":"d4dfb71b399f5d4a958bca9cdfae2be5111801c699e24aebf3f1ea5de4bfc414"} Aug 04 12:10:49.910720 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.910688 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7skwx" event={"ID":"f6453805-e250-4db3-9998-c2ae5888ee09","Type":"ContainerStarted","Data":"3c3aeaff7d5058b8c242991ea31b78411e7e60425ae9507088483b465ae1bade"} Aug 04 12:10:49.910934 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.910779 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7skwx" Aug 04 12:10:49.931605 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:10:49.931557 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7skwx" podStartSLOduration=0.931542789 podStartE2EDuration="931.542789ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:10:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:10:49.930162877 +0000 UTC m=+2262.174882420" watchObservedRunningTime="2026-08-04 12:10:49.931542789 +0000 UTC m=+2262.176262330" Aug 04 12:11:00.917904 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:11:00.917875 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7skwx" Aug 04 12:11:13.089760 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:11:13.089729 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx"] Aug 04 12:11:13.093555 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:11:13.093532 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx" Aug 04 12:11:13.096547 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:11:13.096521 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-f3708437\"" Aug 04 12:11:13.096660 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:11:13.096531 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-f3708437-dockercfg-rbcfm\"" Aug 04 12:11:13.102195 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:11:13.102167 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx"] Aug 04 12:11:13.150368 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:11:13.150326 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04bbd3ee-8c68-4354-a89e-9ff13aebbfb1-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx\" (UID: \"04bbd3ee-8c68-4354-a89e-9ff13aebbfb1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx" Aug 04 12:11:13.150571 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:11:13.150400 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwf8j\" (UniqueName: \"kubernetes.io/projected/04bbd3ee-8c68-4354-a89e-9ff13aebbfb1-kube-api-access-kwf8j\") pod \"payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx\" (UID: \"04bbd3ee-8c68-4354-a89e-9ff13aebbfb1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx" Aug 04 12:11:13.251656 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:11:13.251619 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04bbd3ee-8c68-4354-a89e-9ff13aebbfb1-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx\" (UID: \"04bbd3ee-8c68-4354-a89e-9ff13aebbfb1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx" Aug 04 12:11:13.251901 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:11:13.251677 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwf8j\" (UniqueName: \"kubernetes.io/projected/04bbd3ee-8c68-4354-a89e-9ff13aebbfb1-kube-api-access-kwf8j\") pod \"payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx\" (UID: \"04bbd3ee-8c68-4354-a89e-9ff13aebbfb1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx" Aug 04 12:11:13.252382 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:11:13.252362 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04bbd3ee-8c68-4354-a89e-9ff13aebbfb1-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx\" (UID: \"04bbd3ee-8c68-4354-a89e-9ff13aebbfb1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx" Aug 04 12:11:13.261238 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:11:13.261211 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwf8j\" (UniqueName: \"kubernetes.io/projected/04bbd3ee-8c68-4354-a89e-9ff13aebbfb1-kube-api-access-kwf8j\") pod \"payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx\" (UID: \"04bbd3ee-8c68-4354-a89e-9ff13aebbfb1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx" Aug 04 12:11:13.404205 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:11:13.404107 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx" Aug 04 12:11:13.536702 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:11:13.536670 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx"] Aug 04 12:11:13.538888 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:11:13.538862 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04bbd3ee_8c68_4354_a89e_9ff13aebbfb1.slice/crio-d864d052787d107a2061e71fd7510aed3257408e8256d3ae4f56de69b104046b WatchSource:0}: Error finding container d864d052787d107a2061e71fd7510aed3257408e8256d3ae4f56de69b104046b: Status 404 returned error can't find the container with id d864d052787d107a2061e71fd7510aed3257408e8256d3ae4f56de69b104046b Aug 04 12:11:14.028717 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:11:14.028684 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx" event={"ID":"04bbd3ee-8c68-4354-a89e-9ff13aebbfb1","Type":"ContainerStarted","Data":"8795ae50a57d3ce548dc923f06f53e3cbb3cb031e3770f722259448a96dee235"} Aug 04 12:11:14.028909 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:11:14.028734 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx" event={"ID":"04bbd3ee-8c68-4354-a89e-9ff13aebbfb1","Type":"ContainerStarted","Data":"d864d052787d107a2061e71fd7510aed3257408e8256d3ae4f56de69b104046b"} Aug 04 12:11:14.028909 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:11:14.028798 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx" Aug 04 12:11:14.045337 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:11:14.045277 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx" podStartSLOduration=1.045260679 podStartE2EDuration="1.045260679s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:11:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:11:14.044150675 +0000 UTC m=+2286.288870217" watchObservedRunningTime="2026-08-04 12:11:14.045260679 +0000 UTC m=+2286.289980221" Aug 04 12:11:25.035615 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:11:25.035586 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx" Aug 04 12:13:06.969280 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:13:06.969147 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx"] Aug 04 12:13:06.969885 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:13:06.969754 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx" podUID="04bbd3ee-8c68-4354-a89e-9ff13aebbfb1" containerName="payload-pre-processing" containerID="cri-o://8795ae50a57d3ce548dc923f06f53e3cbb3cb031e3770f722259448a96dee235" gracePeriod=30 Aug 04 12:13:07.222327 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:13:07.222307 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx" Aug 04 12:13:07.266733 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:13:07.266706 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwf8j\" (UniqueName: \"kubernetes.io/projected/04bbd3ee-8c68-4354-a89e-9ff13aebbfb1-kube-api-access-kwf8j\") pod \"04bbd3ee-8c68-4354-a89e-9ff13aebbfb1\" (UID: \"04bbd3ee-8c68-4354-a89e-9ff13aebbfb1\") " Aug 04 12:13:07.266901 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:13:07.266771 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04bbd3ee-8c68-4354-a89e-9ff13aebbfb1-plugins-config-volume\") pod \"04bbd3ee-8c68-4354-a89e-9ff13aebbfb1\" (UID: \"04bbd3ee-8c68-4354-a89e-9ff13aebbfb1\") " Aug 04 12:13:07.267175 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:13:07.267148 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04bbd3ee-8c68-4354-a89e-9ff13aebbfb1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "04bbd3ee-8c68-4354-a89e-9ff13aebbfb1" (UID: "04bbd3ee-8c68-4354-a89e-9ff13aebbfb1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:13:07.269227 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:13:07.269198 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04bbd3ee-8c68-4354-a89e-9ff13aebbfb1-kube-api-access-kwf8j" (OuterVolumeSpecName: "kube-api-access-kwf8j") pod "04bbd3ee-8c68-4354-a89e-9ff13aebbfb1" (UID: "04bbd3ee-8c68-4354-a89e-9ff13aebbfb1"). InnerVolumeSpecName "kube-api-access-kwf8j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:13:07.368180 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:13:07.368145 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04bbd3ee-8c68-4354-a89e-9ff13aebbfb1-plugins-config-volume\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:13:07.368180 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:13:07.368172 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwf8j\" (UniqueName: \"kubernetes.io/projected/04bbd3ee-8c68-4354-a89e-9ff13aebbfb1-kube-api-access-kwf8j\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:13:07.518129 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:13:07.518048 2570 generic.go:356] "Generic (PLEG): container finished" podID="04bbd3ee-8c68-4354-a89e-9ff13aebbfb1" containerID="8795ae50a57d3ce548dc923f06f53e3cbb3cb031e3770f722259448a96dee235" exitCode=0 Aug 04 12:13:07.518129 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:13:07.518091 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx" Aug 04 12:13:07.518299 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:13:07.518135 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx" event={"ID":"04bbd3ee-8c68-4354-a89e-9ff13aebbfb1","Type":"ContainerDied","Data":"8795ae50a57d3ce548dc923f06f53e3cbb3cb031e3770f722259448a96dee235"} Aug 04 12:13:07.518299 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:13:07.518190 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx" event={"ID":"04bbd3ee-8c68-4354-a89e-9ff13aebbfb1","Type":"ContainerDied","Data":"d864d052787d107a2061e71fd7510aed3257408e8256d3ae4f56de69b104046b"} Aug 04 12:13:07.518299 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:13:07.518219 2570 scope.go:117] "RemoveContainer" containerID="8795ae50a57d3ce548dc923f06f53e3cbb3cb031e3770f722259448a96dee235" Aug 04 12:13:07.527102 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:13:07.527084 2570 scope.go:117] "RemoveContainer" containerID="8795ae50a57d3ce548dc923f06f53e3cbb3cb031e3770f722259448a96dee235" Aug 04 12:13:07.527356 ip-10-0-128-47 kubenswrapper[2570]: E0804 12:13:07.527337 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8795ae50a57d3ce548dc923f06f53e3cbb3cb031e3770f722259448a96dee235\": container with ID starting with 8795ae50a57d3ce548dc923f06f53e3cbb3cb031e3770f722259448a96dee235 not found: ID does not exist" containerID="8795ae50a57d3ce548dc923f06f53e3cbb3cb031e3770f722259448a96dee235" Aug 04 12:13:07.527412 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:13:07.527366 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8795ae50a57d3ce548dc923f06f53e3cbb3cb031e3770f722259448a96dee235"} err="failed to get container status \"8795ae50a57d3ce548dc923f06f53e3cbb3cb031e3770f722259448a96dee235\": rpc error: code = NotFound desc = could not find container \"8795ae50a57d3ce548dc923f06f53e3cbb3cb031e3770f722259448a96dee235\": container with ID starting with 8795ae50a57d3ce548dc923f06f53e3cbb3cb031e3770f722259448a96dee235 not found: ID does not exist" Aug 04 12:13:07.540050 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:13:07.540029 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx"] Aug 04 12:13:07.545726 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:13:07.545704 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx"] Aug 04 12:13:08.201725 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:13:08.201692 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04bbd3ee-8c68-4354-a89e-9ff13aebbfb1" path="/var/lib/kubelet/pods/04bbd3ee-8c68-4354-a89e-9ff13aebbfb1/volumes" Aug 04 12:13:08.255368 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:13:08.255341 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 12:13:08.261664 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:13:08.261646 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 12:15:38.262711 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.262676 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl"] Aug 04 12:15:38.263115 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.262996 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04bbd3ee-8c68-4354-a89e-9ff13aebbfb1" containerName="payload-pre-processing" Aug 04 12:15:38.263115 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.263007 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="04bbd3ee-8c68-4354-a89e-9ff13aebbfb1" containerName="payload-pre-processing" Aug 04 12:15:38.263115 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.263078 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="04bbd3ee-8c68-4354-a89e-9ff13aebbfb1" containerName="payload-pre-processing" Aug 04 12:15:38.265962 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.265946 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl" Aug 04 12:15:38.270374 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.270353 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-31323b-dockercfg-56kzz\"" Aug 04 12:15:38.270519 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.270395 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-31323b\"" Aug 04 12:15:38.287785 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.287762 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl"] Aug 04 12:15:38.309328 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.309305 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq"] Aug 04 12:15:38.312457 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.312430 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq" Aug 04 12:15:38.321847 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.321818 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq"] Aug 04 12:15:38.366222 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.366181 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9284d2ed-9ee1-4a5c-be91-f48c029cd948-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl\" (UID: \"9284d2ed-9ee1-4a5c-be91-f48c029cd948\") " pod="openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl" Aug 04 12:15:38.366376 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.366261 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v45q8\" (UniqueName: \"kubernetes.io/projected/9284d2ed-9ee1-4a5c-be91-f48c029cd948-kube-api-access-v45q8\") pod \"payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl\" (UID: \"9284d2ed-9ee1-4a5c-be91-f48c029cd948\") " pod="openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl" Aug 04 12:15:38.467026 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.466990 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9697s\" (UniqueName: \"kubernetes.io/projected/b4cdd7c9-ec91-4447-bf82-3b534b18aa1b-kube-api-access-9697s\") pod \"payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq\" (UID: \"b4cdd7c9-ec91-4447-bf82-3b534b18aa1b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq" Aug 04 12:15:38.467178 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.467034 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9284d2ed-9ee1-4a5c-be91-f48c029cd948-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl\" (UID: \"9284d2ed-9ee1-4a5c-be91-f48c029cd948\") " pod="openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl" Aug 04 12:15:38.467178 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.467065 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v45q8\" (UniqueName: \"kubernetes.io/projected/9284d2ed-9ee1-4a5c-be91-f48c029cd948-kube-api-access-v45q8\") pod \"payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl\" (UID: \"9284d2ed-9ee1-4a5c-be91-f48c029cd948\") " pod="openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl" Aug 04 12:15:38.467178 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.467108 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4cdd7c9-ec91-4447-bf82-3b534b18aa1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq\" (UID: \"b4cdd7c9-ec91-4447-bf82-3b534b18aa1b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq" Aug 04 12:15:38.467721 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.467704 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9284d2ed-9ee1-4a5c-be91-f48c029cd948-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl\" (UID: \"9284d2ed-9ee1-4a5c-be91-f48c029cd948\") " pod="openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl" Aug 04 12:15:38.477326 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.477297 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v45q8\" (UniqueName: \"kubernetes.io/projected/9284d2ed-9ee1-4a5c-be91-f48c029cd948-kube-api-access-v45q8\") pod \"payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl\" (UID: \"9284d2ed-9ee1-4a5c-be91-f48c029cd948\") " pod="openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl" Aug 04 12:15:38.568330 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.568256 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4cdd7c9-ec91-4447-bf82-3b534b18aa1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq\" (UID: \"b4cdd7c9-ec91-4447-bf82-3b534b18aa1b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq" Aug 04 12:15:38.568482 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.568330 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9697s\" (UniqueName: \"kubernetes.io/projected/b4cdd7c9-ec91-4447-bf82-3b534b18aa1b-kube-api-access-9697s\") pod \"payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq\" (UID: \"b4cdd7c9-ec91-4447-bf82-3b534b18aa1b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq" Aug 04 12:15:38.568948 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.568930 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4cdd7c9-ec91-4447-bf82-3b534b18aa1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq\" (UID: \"b4cdd7c9-ec91-4447-bf82-3b534b18aa1b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq" Aug 04 12:15:38.575875 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.575853 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl" Aug 04 12:15:38.576791 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.576771 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9697s\" (UniqueName: \"kubernetes.io/projected/b4cdd7c9-ec91-4447-bf82-3b534b18aa1b-kube-api-access-9697s\") pod \"payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq\" (UID: \"b4cdd7c9-ec91-4447-bf82-3b534b18aa1b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq" Aug 04 12:15:38.624830 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.624606 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq" Aug 04 12:15:38.715571 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.715543 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl"] Aug 04 12:15:38.717917 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:15:38.717886 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9284d2ed_9ee1_4a5c_be91_f48c029cd948.slice/crio-d19688a66ff16f82b23899e544e67dfd71e2c8ccff9647e7ca0d1a90be95629d WatchSource:0}: Error finding container d19688a66ff16f82b23899e544e67dfd71e2c8ccff9647e7ca0d1a90be95629d: Status 404 returned error can't find the container with id d19688a66ff16f82b23899e544e67dfd71e2c8ccff9647e7ca0d1a90be95629d Aug 04 12:15:38.786396 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:38.786373 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq"] Aug 04 12:15:38.789661 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:15:38.789625 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4cdd7c9_ec91_4447_bf82_3b534b18aa1b.slice/crio-7471e033c6bff9c5308a6ab8bd90568bf463068437af4e8bcb90657409120f15 WatchSource:0}: Error finding container 7471e033c6bff9c5308a6ab8bd90568bf463068437af4e8bcb90657409120f15: Status 404 returned error can't find the container with id 7471e033c6bff9c5308a6ab8bd90568bf463068437af4e8bcb90657409120f15 Aug 04 12:15:39.169792 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:39.169701 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl" event={"ID":"9284d2ed-9ee1-4a5c-be91-f48c029cd948","Type":"ContainerStarted","Data":"199336fbfd2c866e2e11095e925c4ec72f31293e91ccc7f04e996d2c3a6ff31e"} Aug 04 12:15:39.169792 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:39.169764 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl" Aug 04 12:15:39.169792 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:39.169789 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl" event={"ID":"9284d2ed-9ee1-4a5c-be91-f48c029cd948","Type":"ContainerStarted","Data":"d19688a66ff16f82b23899e544e67dfd71e2c8ccff9647e7ca0d1a90be95629d"} Aug 04 12:15:39.171240 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:39.171209 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq" event={"ID":"b4cdd7c9-ec91-4447-bf82-3b534b18aa1b","Type":"ContainerStarted","Data":"beb83d3e373653ed92c3b8cb2ba0a651795ceceab7cc9e794fa94e7521fae432"} Aug 04 12:15:39.171353 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:39.171254 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq" event={"ID":"b4cdd7c9-ec91-4447-bf82-3b534b18aa1b","Type":"ContainerStarted","Data":"7471e033c6bff9c5308a6ab8bd90568bf463068437af4e8bcb90657409120f15"} Aug 04 12:15:39.171353 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:39.171304 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq" Aug 04 12:15:39.188694 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:39.188650 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl" podStartSLOduration=1.1886354749999999 podStartE2EDuration="1.188635475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:15:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:15:39.187314982 +0000 UTC m=+2551.432034548" watchObservedRunningTime="2026-08-04 12:15:39.188635475 +0000 UTC m=+2551.433355016" Aug 04 12:15:39.205155 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:39.205113 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq" podStartSLOduration=1.205100156 podStartE2EDuration="1.205100156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:15:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:15:39.203804331 +0000 UTC m=+2551.448523871" watchObservedRunningTime="2026-08-04 12:15:39.205100156 +0000 UTC m=+2551.449819697" Aug 04 12:15:50.178742 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:50.178709 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl" Aug 04 12:15:50.179339 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:15:50.178866 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq" Aug 04 12:17:01.921138 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:01.921046 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-55901c-48njx"] Aug 04 12:17:01.924625 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:01.924603 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-55901c-48njx" Aug 04 12:17:01.927195 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:01.927175 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pv6cr\"" Aug 04 12:17:01.934994 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:01.934966 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-55901c-48njx"] Aug 04 12:17:02.089263 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:02.089229 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d30d587e-43e7-4fbc-8d57-e7951a6adb49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-55901c-48njx\" (UID: \"d30d587e-43e7-4fbc-8d57-e7951a6adb49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-55901c-48njx" Aug 04 12:17:02.190676 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:02.190596 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d30d587e-43e7-4fbc-8d57-e7951a6adb49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-55901c-48njx\" (UID: \"d30d587e-43e7-4fbc-8d57-e7951a6adb49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-55901c-48njx" Aug 04 12:17:02.191199 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:02.191175 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d30d587e-43e7-4fbc-8d57-e7951a6adb49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-55901c-48njx\" (UID: \"d30d587e-43e7-4fbc-8d57-e7951a6adb49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-55901c-48njx" Aug 04 12:17:02.234571 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:02.234548 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-55901c-48njx" Aug 04 12:17:02.373057 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:02.373028 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-55901c-48njx"] Aug 04 12:17:02.375938 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:17:02.375912 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd30d587e_43e7_4fbc_8d57_e7951a6adb49.slice/crio-5b526484a773f6181ea48f7a46b10005d0dd3ef8e8cf934fc130c06622b1259b WatchSource:0}: Error finding container 5b526484a773f6181ea48f7a46b10005d0dd3ef8e8cf934fc130c06622b1259b: Status 404 returned error can't find the container with id 5b526484a773f6181ea48f7a46b10005d0dd3ef8e8cf934fc130c06622b1259b Aug 04 12:17:02.378051 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:02.378034 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 12:17:02.543194 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:02.543166 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-55901c-48njx" event={"ID":"d30d587e-43e7-4fbc-8d57-e7951a6adb49","Type":"ContainerStarted","Data":"5b526484a773f6181ea48f7a46b10005d0dd3ef8e8cf934fc130c06622b1259b"} Aug 04 12:17:05.566061 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:05.566031 2570 generic.go:356] "Generic (PLEG): container finished" podID="d30d587e-43e7-4fbc-8d57-e7951a6adb49" containerID="afec74994fe478d0e14ef0cd2b0924d4302f197d6999a90b456d125511a6a561" exitCode=0 Aug 04 12:17:05.566374 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:05.566069 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-55901c-48njx" event={"ID":"d30d587e-43e7-4fbc-8d57-e7951a6adb49","Type":"ContainerDied","Data":"afec74994fe478d0e14ef0cd2b0924d4302f197d6999a90b456d125511a6a561"} Aug 04 12:17:06.694140 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:06.694119 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-55901c-48njx" Aug 04 12:17:06.833148 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:06.833075 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d30d587e-43e7-4fbc-8d57-e7951a6adb49-maas-api-service-ca\") pod \"d30d587e-43e7-4fbc-8d57-e7951a6adb49\" (UID: \"d30d587e-43e7-4fbc-8d57-e7951a6adb49\") " Aug 04 12:17:06.833400 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:06.833378 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d30d587e-43e7-4fbc-8d57-e7951a6adb49-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d30d587e-43e7-4fbc-8d57-e7951a6adb49" (UID: "d30d587e-43e7-4fbc-8d57-e7951a6adb49"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:17:06.934503 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:06.934477 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d30d587e-43e7-4fbc-8d57-e7951a6adb49-maas-api-service-ca\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:17:07.575696 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:07.575667 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-55901c-48njx" Aug 04 12:17:07.575696 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:07.575691 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-55901c-48njx" event={"ID":"d30d587e-43e7-4fbc-8d57-e7951a6adb49","Type":"ContainerDied","Data":"5b526484a773f6181ea48f7a46b10005d0dd3ef8e8cf934fc130c06622b1259b"} Aug 04 12:17:07.575977 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:07.575738 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b526484a773f6181ea48f7a46b10005d0dd3ef8e8cf934fc130c06622b1259b" Aug 04 12:17:12.571768 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:12.571733 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-55901c-48njx"] Aug 04 12:17:12.577183 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:12.577151 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-55901c-48njx"] Aug 04 12:17:14.195816 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:14.195778 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d30d587e-43e7-4fbc-8d57-e7951a6adb49" path="/var/lib/kubelet/pods/d30d587e-43e7-4fbc-8d57-e7951a6adb49/volumes" Aug 04 12:17:20.571376 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:20.571340 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-31323b-m48wg"] Aug 04 12:17:20.571763 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:20.571715 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d30d587e-43e7-4fbc-8d57-e7951a6adb49" containerName="revoke-keys" Aug 04 12:17:20.571763 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:20.571725 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="d30d587e-43e7-4fbc-8d57-e7951a6adb49" containerName="revoke-keys" Aug 04 12:17:20.571840 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:20.571780 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="d30d587e-43e7-4fbc-8d57-e7951a6adb49" containerName="revoke-keys" Aug 04 12:17:20.574836 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:20.574819 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-31323b-m48wg" Aug 04 12:17:20.579625 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:20.579606 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pv6cr\"" Aug 04 12:17:20.595910 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:20.595888 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-31323b-m48wg"] Aug 04 12:17:20.652955 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:20.652931 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27b8c031-ab93-4bda-a436-02872c3e5909-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-31323b-m48wg\" (UID: \"27b8c031-ab93-4bda-a436-02872c3e5909\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-31323b-m48wg" Aug 04 12:17:20.753950 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:20.753920 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27b8c031-ab93-4bda-a436-02872c3e5909-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-31323b-m48wg\" (UID: \"27b8c031-ab93-4bda-a436-02872c3e5909\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-31323b-m48wg" Aug 04 12:17:20.754506 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:20.754489 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27b8c031-ab93-4bda-a436-02872c3e5909-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-31323b-m48wg\" (UID: \"27b8c031-ab93-4bda-a436-02872c3e5909\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-31323b-m48wg" Aug 04 12:17:20.884316 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:20.884250 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-31323b-m48wg" Aug 04 12:17:21.031355 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:21.031325 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-31323b-m48wg"] Aug 04 12:17:21.033050 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:17:21.033021 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27b8c031_ab93_4bda_a436_02872c3e5909.slice/crio-2aa7581f035903003de8429442ba19d890cdf7418a6093e9330d549bff86bcb1 WatchSource:0}: Error finding container 2aa7581f035903003de8429442ba19d890cdf7418a6093e9330d549bff86bcb1: Status 404 returned error can't find the container with id 2aa7581f035903003de8429442ba19d890cdf7418a6093e9330d549bff86bcb1 Aug 04 12:17:21.643643 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:21.643568 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-31323b-m48wg" event={"ID":"27b8c031-ab93-4bda-a436-02872c3e5909","Type":"ContainerStarted","Data":"9eeca1299c24b4414b25b7030bf96e6ac3f6d188aed12ac1b14be9ffabc76d31"} Aug 04 12:17:21.643643 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:21.643623 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-31323b-m48wg" event={"ID":"27b8c031-ab93-4bda-a436-02872c3e5909","Type":"ContainerStarted","Data":"2aa7581f035903003de8429442ba19d890cdf7418a6093e9330d549bff86bcb1"} Aug 04 12:17:21.673839 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:21.673623 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-31323b-m48wg" podStartSLOduration=1.673603459 podStartE2EDuration="1.673603459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:17:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:17:21.671341881 +0000 UTC m=+2653.916061424" watchObservedRunningTime="2026-08-04 12:17:21.673603459 +0000 UTC m=+2653.918323004" Aug 04 12:17:22.649784 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:22.649752 2570 generic.go:356] "Generic (PLEG): container finished" podID="27b8c031-ab93-4bda-a436-02872c3e5909" containerID="9eeca1299c24b4414b25b7030bf96e6ac3f6d188aed12ac1b14be9ffabc76d31" exitCode=0 Aug 04 12:17:22.650167 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:22.649812 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-31323b-m48wg" event={"ID":"27b8c031-ab93-4bda-a436-02872c3e5909","Type":"ContainerDied","Data":"9eeca1299c24b4414b25b7030bf96e6ac3f6d188aed12ac1b14be9ffabc76d31"} Aug 04 12:17:23.786806 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:23.786782 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-31323b-m48wg" Aug 04 12:17:23.879590 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:23.879562 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27b8c031-ab93-4bda-a436-02872c3e5909-maas-api-service-ca\") pod \"27b8c031-ab93-4bda-a436-02872c3e5909\" (UID: \"27b8c031-ab93-4bda-a436-02872c3e5909\") " Aug 04 12:17:23.879928 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:23.879904 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27b8c031-ab93-4bda-a436-02872c3e5909-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "27b8c031-ab93-4bda-a436-02872c3e5909" (UID: "27b8c031-ab93-4bda-a436-02872c3e5909"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:17:23.980917 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:23.980887 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27b8c031-ab93-4bda-a436-02872c3e5909-maas-api-service-ca\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:17:24.660002 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:24.659976 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-31323b-m48wg" Aug 04 12:17:24.660002 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:24.660000 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-31323b-m48wg" event={"ID":"27b8c031-ab93-4bda-a436-02872c3e5909","Type":"ContainerDied","Data":"2aa7581f035903003de8429442ba19d890cdf7418a6093e9330d549bff86bcb1"} Aug 04 12:17:24.660207 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:24.660047 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2aa7581f035903003de8429442ba19d890cdf7418a6093e9330d549bff86bcb1" Aug 04 12:17:31.286970 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.286938 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl"] Aug 04 12:17:31.287424 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.287140 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl" podUID="9284d2ed-9ee1-4a5c-be91-f48c029cd948" containerName="payload-processing" containerID="cri-o://199336fbfd2c866e2e11095e925c4ec72f31293e91ccc7f04e996d2c3a6ff31e" gracePeriod=30 Aug 04 12:17:31.305915 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.305892 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq"] Aug 04 12:17:31.306152 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.306125 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq" podUID="b4cdd7c9-ec91-4447-bf82-3b534b18aa1b" containerName="payload-pre-processing" containerID="cri-o://beb83d3e373653ed92c3b8cb2ba0a651795ceceab7cc9e794fa94e7521fae432" gracePeriod=30 Aug 04 12:17:31.515840 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.515809 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-31323b-m48wg"] Aug 04 12:17:31.519569 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.519536 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-31323b-m48wg"] Aug 04 12:17:31.572167 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.572143 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl" Aug 04 12:17:31.594923 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.594903 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq" Aug 04 12:17:31.639884 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.639785 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4cdd7c9-ec91-4447-bf82-3b534b18aa1b-plugins-config-volume\") pod \"b4cdd7c9-ec91-4447-bf82-3b534b18aa1b\" (UID: \"b4cdd7c9-ec91-4447-bf82-3b534b18aa1b\") " Aug 04 12:17:31.640087 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.640052 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9697s\" (UniqueName: \"kubernetes.io/projected/b4cdd7c9-ec91-4447-bf82-3b534b18aa1b-kube-api-access-9697s\") pod \"b4cdd7c9-ec91-4447-bf82-3b534b18aa1b\" (UID: \"b4cdd7c9-ec91-4447-bf82-3b534b18aa1b\") " Aug 04 12:17:31.640229 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.640122 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9284d2ed-9ee1-4a5c-be91-f48c029cd948-plugins-config-volume\") pod \"9284d2ed-9ee1-4a5c-be91-f48c029cd948\" (UID: \"9284d2ed-9ee1-4a5c-be91-f48c029cd948\") " Aug 04 12:17:31.640229 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.640146 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v45q8\" (UniqueName: \"kubernetes.io/projected/9284d2ed-9ee1-4a5c-be91-f48c029cd948-kube-api-access-v45q8\") pod \"9284d2ed-9ee1-4a5c-be91-f48c029cd948\" (UID: \"9284d2ed-9ee1-4a5c-be91-f48c029cd948\") " Aug 04 12:17:31.640386 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.640360 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4cdd7c9-ec91-4447-bf82-3b534b18aa1b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b4cdd7c9-ec91-4447-bf82-3b534b18aa1b" (UID: "b4cdd7c9-ec91-4447-bf82-3b534b18aa1b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:17:31.640700 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.640479 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4cdd7c9-ec91-4447-bf82-3b534b18aa1b-plugins-config-volume\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:17:31.640991 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.640951 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9284d2ed-9ee1-4a5c-be91-f48c029cd948-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9284d2ed-9ee1-4a5c-be91-f48c029cd948" (UID: "9284d2ed-9ee1-4a5c-be91-f48c029cd948"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:17:31.642851 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.642829 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9284d2ed-9ee1-4a5c-be91-f48c029cd948-kube-api-access-v45q8" (OuterVolumeSpecName: "kube-api-access-v45q8") pod "9284d2ed-9ee1-4a5c-be91-f48c029cd948" (UID: "9284d2ed-9ee1-4a5c-be91-f48c029cd948"). InnerVolumeSpecName "kube-api-access-v45q8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:17:31.642939 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.642925 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4cdd7c9-ec91-4447-bf82-3b534b18aa1b-kube-api-access-9697s" (OuterVolumeSpecName: "kube-api-access-9697s") pod "b4cdd7c9-ec91-4447-bf82-3b534b18aa1b" (UID: "b4cdd7c9-ec91-4447-bf82-3b534b18aa1b"). InnerVolumeSpecName "kube-api-access-9697s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:17:31.693125 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.693095 2570 generic.go:356] "Generic (PLEG): container finished" podID="9284d2ed-9ee1-4a5c-be91-f48c029cd948" containerID="199336fbfd2c866e2e11095e925c4ec72f31293e91ccc7f04e996d2c3a6ff31e" exitCode=0 Aug 04 12:17:31.693250 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.693139 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl" Aug 04 12:17:31.693250 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.693180 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl" event={"ID":"9284d2ed-9ee1-4a5c-be91-f48c029cd948","Type":"ContainerDied","Data":"199336fbfd2c866e2e11095e925c4ec72f31293e91ccc7f04e996d2c3a6ff31e"} Aug 04 12:17:31.693250 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.693228 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl" event={"ID":"9284d2ed-9ee1-4a5c-be91-f48c029cd948","Type":"ContainerDied","Data":"d19688a66ff16f82b23899e544e67dfd71e2c8ccff9647e7ca0d1a90be95629d"} Aug 04 12:17:31.693477 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.693261 2570 scope.go:117] "RemoveContainer" containerID="199336fbfd2c866e2e11095e925c4ec72f31293e91ccc7f04e996d2c3a6ff31e" Aug 04 12:17:31.694477 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.694432 2570 generic.go:356] "Generic (PLEG): container finished" podID="b4cdd7c9-ec91-4447-bf82-3b534b18aa1b" containerID="beb83d3e373653ed92c3b8cb2ba0a651795ceceab7cc9e794fa94e7521fae432" exitCode=0 Aug 04 12:17:31.694575 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.694499 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq" Aug 04 12:17:31.694575 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.694547 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq" event={"ID":"b4cdd7c9-ec91-4447-bf82-3b534b18aa1b","Type":"ContainerDied","Data":"beb83d3e373653ed92c3b8cb2ba0a651795ceceab7cc9e794fa94e7521fae432"} Aug 04 12:17:31.694640 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.694592 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq" event={"ID":"b4cdd7c9-ec91-4447-bf82-3b534b18aa1b","Type":"ContainerDied","Data":"7471e033c6bff9c5308a6ab8bd90568bf463068437af4e8bcb90657409120f15"} Aug 04 12:17:31.701557 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.701533 2570 scope.go:117] "RemoveContainer" containerID="199336fbfd2c866e2e11095e925c4ec72f31293e91ccc7f04e996d2c3a6ff31e" Aug 04 12:17:31.701805 ip-10-0-128-47 kubenswrapper[2570]: E0804 12:17:31.701782 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"199336fbfd2c866e2e11095e925c4ec72f31293e91ccc7f04e996d2c3a6ff31e\": container with ID starting with 199336fbfd2c866e2e11095e925c4ec72f31293e91ccc7f04e996d2c3a6ff31e not found: ID does not exist" containerID="199336fbfd2c866e2e11095e925c4ec72f31293e91ccc7f04e996d2c3a6ff31e" Aug 04 12:17:31.701857 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.701816 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"199336fbfd2c866e2e11095e925c4ec72f31293e91ccc7f04e996d2c3a6ff31e"} err="failed to get container status \"199336fbfd2c866e2e11095e925c4ec72f31293e91ccc7f04e996d2c3a6ff31e\": rpc error: code = NotFound desc = could not find container \"199336fbfd2c866e2e11095e925c4ec72f31293e91ccc7f04e996d2c3a6ff31e\": container with ID starting with 199336fbfd2c866e2e11095e925c4ec72f31293e91ccc7f04e996d2c3a6ff31e not found: ID does not exist" Aug 04 12:17:31.701857 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.701843 2570 scope.go:117] "RemoveContainer" containerID="beb83d3e373653ed92c3b8cb2ba0a651795ceceab7cc9e794fa94e7521fae432" Aug 04 12:17:31.709527 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.709504 2570 scope.go:117] "RemoveContainer" containerID="beb83d3e373653ed92c3b8cb2ba0a651795ceceab7cc9e794fa94e7521fae432" Aug 04 12:17:31.709764 ip-10-0-128-47 kubenswrapper[2570]: E0804 12:17:31.709747 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"beb83d3e373653ed92c3b8cb2ba0a651795ceceab7cc9e794fa94e7521fae432\": container with ID starting with beb83d3e373653ed92c3b8cb2ba0a651795ceceab7cc9e794fa94e7521fae432 not found: ID does not exist" containerID="beb83d3e373653ed92c3b8cb2ba0a651795ceceab7cc9e794fa94e7521fae432" Aug 04 12:17:31.709821 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.709773 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"beb83d3e373653ed92c3b8cb2ba0a651795ceceab7cc9e794fa94e7521fae432"} err="failed to get container status \"beb83d3e373653ed92c3b8cb2ba0a651795ceceab7cc9e794fa94e7521fae432\": rpc error: code = NotFound desc = could not find container \"beb83d3e373653ed92c3b8cb2ba0a651795ceceab7cc9e794fa94e7521fae432\": container with ID starting with beb83d3e373653ed92c3b8cb2ba0a651795ceceab7cc9e794fa94e7521fae432 not found: ID does not exist" Aug 04 12:17:31.729648 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.729624 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq"] Aug 04 12:17:31.735163 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.735143 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq"] Aug 04 12:17:31.741894 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.741872 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9697s\" (UniqueName: \"kubernetes.io/projected/b4cdd7c9-ec91-4447-bf82-3b534b18aa1b-kube-api-access-9697s\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:17:31.741986 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.741899 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9284d2ed-9ee1-4a5c-be91-f48c029cd948-plugins-config-volume\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:17:31.741986 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.741916 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v45q8\" (UniqueName: \"kubernetes.io/projected/9284d2ed-9ee1-4a5c-be91-f48c029cd948-kube-api-access-v45q8\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:17:31.757983 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.757957 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl"] Aug 04 12:17:31.766467 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:31.766423 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl"] Aug 04 12:17:32.196309 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:32.196277 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27b8c031-ab93-4bda-a436-02872c3e5909" path="/var/lib/kubelet/pods/27b8c031-ab93-4bda-a436-02872c3e5909/volumes" Aug 04 12:17:32.196820 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:32.196797 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9284d2ed-9ee1-4a5c-be91-f48c029cd948" path="/var/lib/kubelet/pods/9284d2ed-9ee1-4a5c-be91-f48c029cd948/volumes" Aug 04 12:17:32.197232 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:32.197214 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4cdd7c9-ec91-4447-bf82-3b534b18aa1b" path="/var/lib/kubelet/pods/b4cdd7c9-ec91-4447-bf82-3b534b18aa1b/volumes" Aug 04 12:17:46.843302 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:46.843268 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj"] Aug 04 12:17:46.843751 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:46.843622 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4cdd7c9-ec91-4447-bf82-3b534b18aa1b" containerName="payload-pre-processing" Aug 04 12:17:46.843751 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:46.843635 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4cdd7c9-ec91-4447-bf82-3b534b18aa1b" containerName="payload-pre-processing" Aug 04 12:17:46.843751 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:46.843647 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9284d2ed-9ee1-4a5c-be91-f48c029cd948" containerName="payload-processing" Aug 04 12:17:46.843751 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:46.843653 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="9284d2ed-9ee1-4a5c-be91-f48c029cd948" containerName="payload-processing" Aug 04 12:17:46.843751 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:46.843661 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27b8c031-ab93-4bda-a436-02872c3e5909" containerName="revoke-keys" Aug 04 12:17:46.843751 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:46.843667 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="27b8c031-ab93-4bda-a436-02872c3e5909" containerName="revoke-keys" Aug 04 12:17:46.843751 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:46.843739 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="27b8c031-ab93-4bda-a436-02872c3e5909" containerName="revoke-keys" Aug 04 12:17:46.843751 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:46.843747 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="9284d2ed-9ee1-4a5c-be91-f48c029cd948" containerName="payload-processing" Aug 04 12:17:46.843751 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:46.843752 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4cdd7c9-ec91-4447-bf82-3b534b18aa1b" containerName="payload-pre-processing" Aug 04 12:17:46.846760 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:46.846738 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj" Aug 04 12:17:46.852363 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:46.852339 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-62afff\"" Aug 04 12:17:46.852363 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:46.852355 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-62afff-dockercfg-8nmtd\"" Aug 04 12:17:46.857901 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:46.857879 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj"] Aug 04 12:17:46.960665 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:46.960636 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vrj4\" (UniqueName: \"kubernetes.io/projected/0b85c0f6-caee-4103-aeff-7060e0cffac6-kube-api-access-2vrj4\") pod \"payload-processing-e2e-api-a-62afff-555987f44b-fhchj\" (UID: \"0b85c0f6-caee-4103-aeff-7060e0cffac6\") " pod="openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj" Aug 04 12:17:46.960780 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:46.960700 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b85c0f6-caee-4103-aeff-7060e0cffac6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-62afff-555987f44b-fhchj\" (UID: \"0b85c0f6-caee-4103-aeff-7060e0cffac6\") " pod="openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj" Aug 04 12:17:47.061561 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:47.061535 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vrj4\" (UniqueName: \"kubernetes.io/projected/0b85c0f6-caee-4103-aeff-7060e0cffac6-kube-api-access-2vrj4\") pod \"payload-processing-e2e-api-a-62afff-555987f44b-fhchj\" (UID: \"0b85c0f6-caee-4103-aeff-7060e0cffac6\") " pod="openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj" Aug 04 12:17:47.061679 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:47.061598 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b85c0f6-caee-4103-aeff-7060e0cffac6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-62afff-555987f44b-fhchj\" (UID: \"0b85c0f6-caee-4103-aeff-7060e0cffac6\") " pod="openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj" Aug 04 12:17:47.062218 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:47.062200 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b85c0f6-caee-4103-aeff-7060e0cffac6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-62afff-555987f44b-fhchj\" (UID: \"0b85c0f6-caee-4103-aeff-7060e0cffac6\") " pod="openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj" Aug 04 12:17:47.072008 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:47.071986 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vrj4\" (UniqueName: \"kubernetes.io/projected/0b85c0f6-caee-4103-aeff-7060e0cffac6-kube-api-access-2vrj4\") pod \"payload-processing-e2e-api-a-62afff-555987f44b-fhchj\" (UID: \"0b85c0f6-caee-4103-aeff-7060e0cffac6\") " pod="openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj" Aug 04 12:17:47.157361 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:47.157305 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj" Aug 04 12:17:47.321422 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:47.321394 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj"] Aug 04 12:17:47.322974 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:17:47.322950 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b85c0f6_caee_4103_aeff_7060e0cffac6.slice/crio-28f72248cebd546fcd031f79ec68580173267dfa3d5b18a0e2b887360d7a3ef2 WatchSource:0}: Error finding container 28f72248cebd546fcd031f79ec68580173267dfa3d5b18a0e2b887360d7a3ef2: Status 404 returned error can't find the container with id 28f72248cebd546fcd031f79ec68580173267dfa3d5b18a0e2b887360d7a3ef2 Aug 04 12:17:47.768097 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:47.768044 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj" event={"ID":"0b85c0f6-caee-4103-aeff-7060e0cffac6","Type":"ContainerStarted","Data":"a7ad89de8ef695df2919dc2d1336e1b6d1dffbeacfae2d7ad6bae427008dd692"} Aug 04 12:17:47.768271 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:47.768116 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj" event={"ID":"0b85c0f6-caee-4103-aeff-7060e0cffac6","Type":"ContainerStarted","Data":"28f72248cebd546fcd031f79ec68580173267dfa3d5b18a0e2b887360d7a3ef2"} Aug 04 12:17:47.768271 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:47.768146 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj" Aug 04 12:17:47.801357 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:47.801306 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj" podStartSLOduration=1.801290776 podStartE2EDuration="1.801290776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:17:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:17:47.800176636 +0000 UTC m=+2680.044896180" watchObservedRunningTime="2026-08-04 12:17:47.801290776 +0000 UTC m=+2680.046010332" Aug 04 12:17:58.774498 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:17:58.774440 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj" Aug 04 12:18:08.288559 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:08.288489 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 12:18:08.294698 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:08.294677 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 12:18:41.836072 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:41.836038 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62afff-w68zm"] Aug 04 12:18:41.839832 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:41.839806 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62afff-w68zm" Aug 04 12:18:41.843054 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:41.843030 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pv6cr\"" Aug 04 12:18:41.854925 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:41.854900 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62afff-w68zm"] Aug 04 12:18:41.878865 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:41.878838 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e752686-fd49-4b36-9374-a75c63c42393-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-62afff-w68zm\" (UID: \"9e752686-fd49-4b36-9374-a75c63c42393\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62afff-w68zm" Aug 04 12:18:41.979781 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:41.979757 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e752686-fd49-4b36-9374-a75c63c42393-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-62afff-w68zm\" (UID: \"9e752686-fd49-4b36-9374-a75c63c42393\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62afff-w68zm" Aug 04 12:18:41.980351 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:41.980334 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e752686-fd49-4b36-9374-a75c63c42393-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-62afff-w68zm\" (UID: \"9e752686-fd49-4b36-9374-a75c63c42393\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62afff-w68zm" Aug 04 12:18:42.149858 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:42.149772 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62afff-w68zm" Aug 04 12:18:42.288064 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:42.288041 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62afff-w68zm"] Aug 04 12:18:42.290491 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:18:42.290464 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e752686_fd49_4b36_9374_a75c63c42393.slice/crio-9c8fc1bb3e7b9dc3668db774cac6e3882bc9cea45a89b2bf91631012a41277a8 WatchSource:0}: Error finding container 9c8fc1bb3e7b9dc3668db774cac6e3882bc9cea45a89b2bf91631012a41277a8: Status 404 returned error can't find the container with id 9c8fc1bb3e7b9dc3668db774cac6e3882bc9cea45a89b2bf91631012a41277a8 Aug 04 12:18:43.013270 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:43.013238 2570 generic.go:356] "Generic (PLEG): container finished" podID="9e752686-fd49-4b36-9374-a75c63c42393" containerID="8249b88d943b90c0519a7d9f64feef2d69f4667a877ff549a907b850b020fb9e" exitCode=0 Aug 04 12:18:43.013603 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:43.013294 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62afff-w68zm" event={"ID":"9e752686-fd49-4b36-9374-a75c63c42393","Type":"ContainerDied","Data":"8249b88d943b90c0519a7d9f64feef2d69f4667a877ff549a907b850b020fb9e"} Aug 04 12:18:43.013603 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:43.013360 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62afff-w68zm" event={"ID":"9e752686-fd49-4b36-9374-a75c63c42393","Type":"ContainerStarted","Data":"9c8fc1bb3e7b9dc3668db774cac6e3882bc9cea45a89b2bf91631012a41277a8"} Aug 04 12:18:44.167545 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:44.167520 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62afff-w68zm" Aug 04 12:18:44.198288 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:44.198261 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e752686-fd49-4b36-9374-a75c63c42393-maas-api-service-ca\") pod \"9e752686-fd49-4b36-9374-a75c63c42393\" (UID: \"9e752686-fd49-4b36-9374-a75c63c42393\") " Aug 04 12:18:44.198637 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:44.198612 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e752686-fd49-4b36-9374-a75c63c42393-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9e752686-fd49-4b36-9374-a75c63c42393" (UID: "9e752686-fd49-4b36-9374-a75c63c42393"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:18:44.299650 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:44.299590 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e752686-fd49-4b36-9374-a75c63c42393-maas-api-service-ca\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:18:45.023786 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:45.023746 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62afff-w68zm" event={"ID":"9e752686-fd49-4b36-9374-a75c63c42393","Type":"ContainerDied","Data":"9c8fc1bb3e7b9dc3668db774cac6e3882bc9cea45a89b2bf91631012a41277a8"} Aug 04 12:18:45.023973 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:45.023799 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c8fc1bb3e7b9dc3668db774cac6e3882bc9cea45a89b2bf91631012a41277a8" Aug 04 12:18:45.023973 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:45.023757 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62afff-w68zm" Aug 04 12:18:57.506107 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:57.506068 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj"] Aug 04 12:18:57.506564 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:57.506299 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj" podUID="0b85c0f6-caee-4103-aeff-7060e0cffac6" containerName="payload-processing" containerID="cri-o://a7ad89de8ef695df2919dc2d1336e1b6d1dffbeacfae2d7ad6bae427008dd692" gracePeriod=30 Aug 04 12:18:57.753793 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:57.753769 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj" Aug 04 12:18:57.813424 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:57.813342 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vrj4\" (UniqueName: \"kubernetes.io/projected/0b85c0f6-caee-4103-aeff-7060e0cffac6-kube-api-access-2vrj4\") pod \"0b85c0f6-caee-4103-aeff-7060e0cffac6\" (UID: \"0b85c0f6-caee-4103-aeff-7060e0cffac6\") " Aug 04 12:18:57.813424 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:57.813374 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b85c0f6-caee-4103-aeff-7060e0cffac6-plugins-config-volume\") pod \"0b85c0f6-caee-4103-aeff-7060e0cffac6\" (UID: \"0b85c0f6-caee-4103-aeff-7060e0cffac6\") " Aug 04 12:18:57.813768 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:57.813745 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b85c0f6-caee-4103-aeff-7060e0cffac6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0b85c0f6-caee-4103-aeff-7060e0cffac6" (UID: "0b85c0f6-caee-4103-aeff-7060e0cffac6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:18:57.815434 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:57.815412 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b85c0f6-caee-4103-aeff-7060e0cffac6-kube-api-access-2vrj4" (OuterVolumeSpecName: "kube-api-access-2vrj4") pod "0b85c0f6-caee-4103-aeff-7060e0cffac6" (UID: "0b85c0f6-caee-4103-aeff-7060e0cffac6"). InnerVolumeSpecName "kube-api-access-2vrj4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:18:57.847734 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:57.847701 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62afff-w68zm"] Aug 04 12:18:57.849321 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:57.849293 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62afff-w68zm"] Aug 04 12:18:57.914729 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:57.914695 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vrj4\" (UniqueName: \"kubernetes.io/projected/0b85c0f6-caee-4103-aeff-7060e0cffac6-kube-api-access-2vrj4\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:18:57.914729 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:57.914724 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b85c0f6-caee-4103-aeff-7060e0cffac6-plugins-config-volume\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:18:58.081276 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:58.081186 2570 generic.go:356] "Generic (PLEG): container finished" podID="0b85c0f6-caee-4103-aeff-7060e0cffac6" containerID="a7ad89de8ef695df2919dc2d1336e1b6d1dffbeacfae2d7ad6bae427008dd692" exitCode=0 Aug 04 12:18:58.081276 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:58.081229 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj" Aug 04 12:18:58.081532 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:58.081232 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj" event={"ID":"0b85c0f6-caee-4103-aeff-7060e0cffac6","Type":"ContainerDied","Data":"a7ad89de8ef695df2919dc2d1336e1b6d1dffbeacfae2d7ad6bae427008dd692"} Aug 04 12:18:58.081532 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:58.081375 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj" event={"ID":"0b85c0f6-caee-4103-aeff-7060e0cffac6","Type":"ContainerDied","Data":"28f72248cebd546fcd031f79ec68580173267dfa3d5b18a0e2b887360d7a3ef2"} Aug 04 12:18:58.081532 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:58.081413 2570 scope.go:117] "RemoveContainer" containerID="a7ad89de8ef695df2919dc2d1336e1b6d1dffbeacfae2d7ad6bae427008dd692" Aug 04 12:18:58.089982 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:58.089964 2570 scope.go:117] "RemoveContainer" containerID="a7ad89de8ef695df2919dc2d1336e1b6d1dffbeacfae2d7ad6bae427008dd692" Aug 04 12:18:58.090268 ip-10-0-128-47 kubenswrapper[2570]: E0804 12:18:58.090249 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7ad89de8ef695df2919dc2d1336e1b6d1dffbeacfae2d7ad6bae427008dd692\": container with ID starting with a7ad89de8ef695df2919dc2d1336e1b6d1dffbeacfae2d7ad6bae427008dd692 not found: ID does not exist" containerID="a7ad89de8ef695df2919dc2d1336e1b6d1dffbeacfae2d7ad6bae427008dd692" Aug 04 12:18:58.090322 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:58.090277 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7ad89de8ef695df2919dc2d1336e1b6d1dffbeacfae2d7ad6bae427008dd692"} err="failed to get container status \"a7ad89de8ef695df2919dc2d1336e1b6d1dffbeacfae2d7ad6bae427008dd692\": rpc error: code = NotFound desc = could not find container \"a7ad89de8ef695df2919dc2d1336e1b6d1dffbeacfae2d7ad6bae427008dd692\": container with ID starting with a7ad89de8ef695df2919dc2d1336e1b6d1dffbeacfae2d7ad6bae427008dd692 not found: ID does not exist" Aug 04 12:18:58.103717 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:58.103693 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj"] Aug 04 12:18:58.109324 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:58.109299 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj"] Aug 04 12:18:58.195193 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:58.195146 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b85c0f6-caee-4103-aeff-7060e0cffac6" path="/var/lib/kubelet/pods/0b85c0f6-caee-4103-aeff-7060e0cffac6/volumes" Aug 04 12:18:58.195513 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:18:58.195497 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e752686-fd49-4b36-9374-a75c63c42393" path="/var/lib/kubelet/pods/9e752686-fd49-4b36-9374-a75c63c42393/volumes" Aug 04 12:19:05.160142 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:05.160102 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e17b5-477nj"] Aug 04 12:19:05.160570 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:05.160474 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b85c0f6-caee-4103-aeff-7060e0cffac6" containerName="payload-processing" Aug 04 12:19:05.160570 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:05.160489 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b85c0f6-caee-4103-aeff-7060e0cffac6" containerName="payload-processing" Aug 04 12:19:05.160570 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:05.160511 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e752686-fd49-4b36-9374-a75c63c42393" containerName="revoke-keys" Aug 04 12:19:05.160570 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:05.160517 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e752686-fd49-4b36-9374-a75c63c42393" containerName="revoke-keys" Aug 04 12:19:05.160570 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:05.160564 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e752686-fd49-4b36-9374-a75c63c42393" containerName="revoke-keys" Aug 04 12:19:05.160762 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:05.160575 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b85c0f6-caee-4103-aeff-7060e0cffac6" containerName="payload-processing" Aug 04 12:19:05.165184 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:05.165163 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e17b5-477nj" Aug 04 12:19:05.168281 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:05.168264 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pv6cr\"" Aug 04 12:19:05.175964 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:05.175941 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e17b5-477nj"] Aug 04 12:19:05.272818 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:05.272788 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a8f82d9-9319-40b7-a096-8dbc343a98c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9e17b5-477nj\" (UID: \"3a8f82d9-9319-40b7-a096-8dbc343a98c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e17b5-477nj" Aug 04 12:19:05.374192 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:05.374158 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a8f82d9-9319-40b7-a096-8dbc343a98c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9e17b5-477nj\" (UID: \"3a8f82d9-9319-40b7-a096-8dbc343a98c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e17b5-477nj" Aug 04 12:19:05.375267 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:05.375234 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a8f82d9-9319-40b7-a096-8dbc343a98c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9e17b5-477nj\" (UID: \"3a8f82d9-9319-40b7-a096-8dbc343a98c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e17b5-477nj" Aug 04 12:19:05.475591 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:05.475560 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e17b5-477nj" Aug 04 12:19:05.612611 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:05.612582 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e17b5-477nj"] Aug 04 12:19:05.614881 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:19:05.614847 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a8f82d9_9319_40b7_a096_8dbc343a98c8.slice/crio-932ed8751df4a280f574773509eab244f26c322840aedde18bbf9012af59f242 WatchSource:0}: Error finding container 932ed8751df4a280f574773509eab244f26c322840aedde18bbf9012af59f242: Status 404 returned error can't find the container with id 932ed8751df4a280f574773509eab244f26c322840aedde18bbf9012af59f242 Aug 04 12:19:06.117899 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:06.117819 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e17b5-477nj" event={"ID":"3a8f82d9-9319-40b7-a096-8dbc343a98c8","Type":"ContainerStarted","Data":"6930c01b6aba2d8c31f5414c7babeabdc1370abd43e9dc4d26f8a2279bf1d406"} Aug 04 12:19:06.117899 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:06.117869 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e17b5-477nj" event={"ID":"3a8f82d9-9319-40b7-a096-8dbc343a98c8","Type":"ContainerStarted","Data":"932ed8751df4a280f574773509eab244f26c322840aedde18bbf9012af59f242"} Aug 04 12:19:06.139059 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:06.139012 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e17b5-477nj" podStartSLOduration=1.138998099 podStartE2EDuration="1.138998099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:19:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:19:06.137812763 +0000 UTC m=+2758.382532306" watchObservedRunningTime="2026-08-04 12:19:06.138998099 +0000 UTC m=+2758.383717641" Aug 04 12:19:07.124018 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:07.123980 2570 generic.go:356] "Generic (PLEG): container finished" podID="3a8f82d9-9319-40b7-a096-8dbc343a98c8" containerID="6930c01b6aba2d8c31f5414c7babeabdc1370abd43e9dc4d26f8a2279bf1d406" exitCode=0 Aug 04 12:19:07.124413 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:07.124040 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e17b5-477nj" event={"ID":"3a8f82d9-9319-40b7-a096-8dbc343a98c8","Type":"ContainerDied","Data":"6930c01b6aba2d8c31f5414c7babeabdc1370abd43e9dc4d26f8a2279bf1d406"} Aug 04 12:19:08.254074 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:08.254051 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e17b5-477nj" Aug 04 12:19:08.399407 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:08.399382 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a8f82d9-9319-40b7-a096-8dbc343a98c8-maas-api-service-ca\") pod \"3a8f82d9-9319-40b7-a096-8dbc343a98c8\" (UID: \"3a8f82d9-9319-40b7-a096-8dbc343a98c8\") " Aug 04 12:19:08.399696 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:08.399676 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a8f82d9-9319-40b7-a096-8dbc343a98c8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3a8f82d9-9319-40b7-a096-8dbc343a98c8" (UID: "3a8f82d9-9319-40b7-a096-8dbc343a98c8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:19:08.500287 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:08.500256 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a8f82d9-9319-40b7-a096-8dbc343a98c8-maas-api-service-ca\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:19:09.134430 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:09.134405 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e17b5-477nj" Aug 04 12:19:09.134587 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:09.134407 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e17b5-477nj" event={"ID":"3a8f82d9-9319-40b7-a096-8dbc343a98c8","Type":"ContainerDied","Data":"932ed8751df4a280f574773509eab244f26c322840aedde18bbf9012af59f242"} Aug 04 12:19:09.134587 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:09.134491 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="932ed8751df4a280f574773509eab244f26c322840aedde18bbf9012af59f242" Aug 04 12:19:21.052748 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:21.052711 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e17b5-477nj"] Aug 04 12:19:21.058380 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:21.058355 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e17b5-477nj"] Aug 04 12:19:22.194614 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:22.194578 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a8f82d9-9319-40b7-a096-8dbc343a98c8" path="/var/lib/kubelet/pods/3a8f82d9-9319-40b7-a096-8dbc343a98c8/volumes" Aug 04 12:19:34.815344 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:34.815262 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs"] Aug 04 12:19:34.816312 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:34.815653 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a8f82d9-9319-40b7-a096-8dbc343a98c8" containerName="revoke-keys" Aug 04 12:19:34.816312 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:34.815666 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a8f82d9-9319-40b7-a096-8dbc343a98c8" containerName="revoke-keys" Aug 04 12:19:34.816312 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:34.815733 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a8f82d9-9319-40b7-a096-8dbc343a98c8" containerName="revoke-keys" Aug 04 12:19:34.818863 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:34.818847 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs" Aug 04 12:19:34.821828 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:34.821808 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-a7013e-dockercfg-rn9vh\"" Aug 04 12:19:34.822124 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:34.822107 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-a7013e\"" Aug 04 12:19:34.851844 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:34.851820 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs"] Aug 04 12:19:34.928968 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:34.928937 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7d6c595-2807-4fca-9dcb-96d66bd9682a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs\" (UID: \"a7d6c595-2807-4fca-9dcb-96d66bd9682a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs" Aug 04 12:19:34.929118 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:34.929053 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44vnv\" (UniqueName: \"kubernetes.io/projected/a7d6c595-2807-4fca-9dcb-96d66bd9682a-kube-api-access-44vnv\") pod \"payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs\" (UID: \"a7d6c595-2807-4fca-9dcb-96d66bd9682a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs" Aug 04 12:19:35.029978 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:35.029942 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44vnv\" (UniqueName: \"kubernetes.io/projected/a7d6c595-2807-4fca-9dcb-96d66bd9682a-kube-api-access-44vnv\") pod \"payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs\" (UID: \"a7d6c595-2807-4fca-9dcb-96d66bd9682a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs" Aug 04 12:19:35.030152 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:35.030025 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7d6c595-2807-4fca-9dcb-96d66bd9682a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs\" (UID: \"a7d6c595-2807-4fca-9dcb-96d66bd9682a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs" Aug 04 12:19:35.030682 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:35.030664 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7d6c595-2807-4fca-9dcb-96d66bd9682a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs\" (UID: \"a7d6c595-2807-4fca-9dcb-96d66bd9682a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs" Aug 04 12:19:35.038805 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:35.038784 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44vnv\" (UniqueName: \"kubernetes.io/projected/a7d6c595-2807-4fca-9dcb-96d66bd9682a-kube-api-access-44vnv\") pod \"payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs\" (UID: \"a7d6c595-2807-4fca-9dcb-96d66bd9682a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs" Aug 04 12:19:35.128978 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:35.128896 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs" Aug 04 12:19:35.463475 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:35.463430 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs"] Aug 04 12:19:35.468479 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:19:35.468427 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7d6c595_2807_4fca_9dcb_96d66bd9682a.slice/crio-81977c5142f85d73bfccab84f5b9d2f833a4adb2e06b0ed96c4fb359d821d8ba WatchSource:0}: Error finding container 81977c5142f85d73bfccab84f5b9d2f833a4adb2e06b0ed96c4fb359d821d8ba: Status 404 returned error can't find the container with id 81977c5142f85d73bfccab84f5b9d2f833a4adb2e06b0ed96c4fb359d821d8ba Aug 04 12:19:36.257778 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:36.257746 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs" event={"ID":"a7d6c595-2807-4fca-9dcb-96d66bd9682a","Type":"ContainerStarted","Data":"9cae0ac98d13edfdf8f018767af334e4145570e5bca13d653e347c799444bc2e"} Aug 04 12:19:36.258122 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:36.257789 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs" event={"ID":"a7d6c595-2807-4fca-9dcb-96d66bd9682a","Type":"ContainerStarted","Data":"81977c5142f85d73bfccab84f5b9d2f833a4adb2e06b0ed96c4fb359d821d8ba"} Aug 04 12:19:36.258122 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:36.257851 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs" Aug 04 12:19:36.276346 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:36.276300 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs" podStartSLOduration=2.276286496 podStartE2EDuration="2.276286496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:19:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:19:36.275553347 +0000 UTC m=+2788.520272893" watchObservedRunningTime="2026-08-04 12:19:36.276286496 +0000 UTC m=+2788.521006035" Aug 04 12:19:47.265252 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:19:47.265224 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs" Aug 04 12:20:03.908658 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:20:03.908620 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49"] Aug 04 12:20:03.912067 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:20:03.912045 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49" Aug 04 12:20:03.914701 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:20:03.914677 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-038ca6-dockercfg-pb5fb\"" Aug 04 12:20:03.914813 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:20:03.914750 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-038ca6\"" Aug 04 12:20:03.924504 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:20:03.924480 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49"] Aug 04 12:20:03.983865 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:20:03.983837 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2515f6d0-9c47-4c2f-9a99-bdb975c8884c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49\" (UID: \"2515f6d0-9c47-4c2f-9a99-bdb975c8884c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49" Aug 04 12:20:03.983991 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:20:03.983868 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhpzk\" (UniqueName: \"kubernetes.io/projected/2515f6d0-9c47-4c2f-9a99-bdb975c8884c-kube-api-access-vhpzk\") pod \"payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49\" (UID: \"2515f6d0-9c47-4c2f-9a99-bdb975c8884c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49" Aug 04 12:20:04.084489 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:20:04.084438 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2515f6d0-9c47-4c2f-9a99-bdb975c8884c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49\" (UID: \"2515f6d0-9c47-4c2f-9a99-bdb975c8884c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49" Aug 04 12:20:04.084489 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:20:04.084492 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhpzk\" (UniqueName: \"kubernetes.io/projected/2515f6d0-9c47-4c2f-9a99-bdb975c8884c-kube-api-access-vhpzk\") pod \"payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49\" (UID: \"2515f6d0-9c47-4c2f-9a99-bdb975c8884c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49" Aug 04 12:20:04.085165 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:20:04.085146 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2515f6d0-9c47-4c2f-9a99-bdb975c8884c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49\" (UID: \"2515f6d0-9c47-4c2f-9a99-bdb975c8884c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49" Aug 04 12:20:04.094032 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:20:04.094007 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhpzk\" (UniqueName: \"kubernetes.io/projected/2515f6d0-9c47-4c2f-9a99-bdb975c8884c-kube-api-access-vhpzk\") pod \"payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49\" (UID: \"2515f6d0-9c47-4c2f-9a99-bdb975c8884c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49" Aug 04 12:20:04.222205 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:20:04.222181 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49" Aug 04 12:20:04.359127 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:20:04.359102 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49"] Aug 04 12:20:04.360943 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:20:04.360917 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2515f6d0_9c47_4c2f_9a99_bdb975c8884c.slice/crio-2f224097bd089d880d6f80985b0f9cc6eb9c582069b55d6db0fee27f34b7136b WatchSource:0}: Error finding container 2f224097bd089d880d6f80985b0f9cc6eb9c582069b55d6db0fee27f34b7136b: Status 404 returned error can't find the container with id 2f224097bd089d880d6f80985b0f9cc6eb9c582069b55d6db0fee27f34b7136b Aug 04 12:20:04.386839 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:20:04.386814 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49" event={"ID":"2515f6d0-9c47-4c2f-9a99-bdb975c8884c","Type":"ContainerStarted","Data":"2f224097bd089d880d6f80985b0f9cc6eb9c582069b55d6db0fee27f34b7136b"} Aug 04 12:20:05.392600 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:20:05.392565 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49" event={"ID":"2515f6d0-9c47-4c2f-9a99-bdb975c8884c","Type":"ContainerStarted","Data":"58ef048ea7b1c9d2a7f8445b80653e9e4046cdc955f3372287248ae474b4d2df"} Aug 04 12:20:05.392995 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:20:05.392645 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49" Aug 04 12:20:05.420964 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:20:05.420918 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49" podStartSLOduration=2.420903624 podStartE2EDuration="2.420903624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:20:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:20:05.418823727 +0000 UTC m=+2817.663543291" watchObservedRunningTime="2026-08-04 12:20:05.420903624 +0000 UTC m=+2817.665623165" Aug 04 12:20:16.399620 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:20:16.399586 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49" Aug 04 12:23:08.320968 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:23:08.320943 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 12:23:08.327343 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:23:08.327323 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 12:23:08.415775 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:23:08.415746 2570 scope.go:117] "RemoveContainer" containerID="afec74994fe478d0e14ef0cd2b0924d4302f197d6999a90b456d125511a6a561" Aug 04 12:24:08.435625 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:24:08.435544 2570 scope.go:117] "RemoveContainer" containerID="9eeca1299c24b4414b25b7030bf96e6ac3f6d188aed12ac1b14be9ffabc76d31" Aug 04 12:25:08.454164 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:25:08.454132 2570 scope.go:117] "RemoveContainer" containerID="6930c01b6aba2d8c31f5414c7babeabdc1370abd43e9dc4d26f8a2279bf1d406" Aug 04 12:25:08.462018 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:25:08.461993 2570 scope.go:117] "RemoveContainer" containerID="8249b88d943b90c0519a7d9f64feef2d69f4667a877ff549a907b850b020fb9e" Aug 04 12:28:08.348609 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:28:08.348577 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 12:28:08.355636 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:28:08.355618 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 12:29:45.287943 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.287907 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h"] Aug 04 12:29:45.291052 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.291035 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h" Aug 04 12:29:45.295029 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.295007 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-a888a4-dockercfg-4thj5\"" Aug 04 12:29:45.295165 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.295120 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-a888a4\"" Aug 04 12:29:45.310782 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.310760 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d"] Aug 04 12:29:45.314471 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.314432 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h"] Aug 04 12:29:45.314562 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.314550 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d" Aug 04 12:29:45.339241 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.339221 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d"] Aug 04 12:29:45.455052 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.455021 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2nmk\" (UniqueName: \"kubernetes.io/projected/ddfb6594-e3fd-487a-9556-f1c779926ec0-kube-api-access-k2nmk\") pod \"payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d\" (UID: \"ddfb6594-e3fd-487a-9556-f1c779926ec0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d" Aug 04 12:29:45.455220 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.455071 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd830dcf-41a3-423d-8e54-6d9438ee0db9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h\" (UID: \"fd830dcf-41a3-423d-8e54-6d9438ee0db9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h" Aug 04 12:29:45.455220 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.455099 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqlc6\" (UniqueName: \"kubernetes.io/projected/fd830dcf-41a3-423d-8e54-6d9438ee0db9-kube-api-access-bqlc6\") pod \"payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h\" (UID: \"fd830dcf-41a3-423d-8e54-6d9438ee0db9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h" Aug 04 12:29:45.455220 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.455187 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddfb6594-e3fd-487a-9556-f1c779926ec0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d\" (UID: \"ddfb6594-e3fd-487a-9556-f1c779926ec0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d" Aug 04 12:29:45.555879 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.555812 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddfb6594-e3fd-487a-9556-f1c779926ec0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d\" (UID: \"ddfb6594-e3fd-487a-9556-f1c779926ec0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d" Aug 04 12:29:45.555879 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.555873 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2nmk\" (UniqueName: \"kubernetes.io/projected/ddfb6594-e3fd-487a-9556-f1c779926ec0-kube-api-access-k2nmk\") pod \"payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d\" (UID: \"ddfb6594-e3fd-487a-9556-f1c779926ec0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d" Aug 04 12:29:45.556091 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.555893 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd830dcf-41a3-423d-8e54-6d9438ee0db9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h\" (UID: \"fd830dcf-41a3-423d-8e54-6d9438ee0db9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h" Aug 04 12:29:45.556091 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.555917 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqlc6\" (UniqueName: \"kubernetes.io/projected/fd830dcf-41a3-423d-8e54-6d9438ee0db9-kube-api-access-bqlc6\") pod \"payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h\" (UID: \"fd830dcf-41a3-423d-8e54-6d9438ee0db9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h" Aug 04 12:29:45.556536 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.556517 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddfb6594-e3fd-487a-9556-f1c779926ec0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d\" (UID: \"ddfb6594-e3fd-487a-9556-f1c779926ec0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d" Aug 04 12:29:45.556639 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.556622 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd830dcf-41a3-423d-8e54-6d9438ee0db9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h\" (UID: \"fd830dcf-41a3-423d-8e54-6d9438ee0db9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h" Aug 04 12:29:45.567460 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.567419 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqlc6\" (UniqueName: \"kubernetes.io/projected/fd830dcf-41a3-423d-8e54-6d9438ee0db9-kube-api-access-bqlc6\") pod \"payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h\" (UID: \"fd830dcf-41a3-423d-8e54-6d9438ee0db9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h" Aug 04 12:29:45.567589 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.567503 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2nmk\" (UniqueName: \"kubernetes.io/projected/ddfb6594-e3fd-487a-9556-f1c779926ec0-kube-api-access-k2nmk\") pod \"payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d\" (UID: \"ddfb6594-e3fd-487a-9556-f1c779926ec0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d" Aug 04 12:29:45.600361 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.600332 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h" Aug 04 12:29:45.624046 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.624027 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d" Aug 04 12:29:45.747700 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.747669 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h"] Aug 04 12:29:45.750068 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:29:45.750039 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd830dcf_41a3_423d_8e54_6d9438ee0db9.slice/crio-cde99ac77f9fbcc2167df80afb15cb4423e7cfb4e0708c81b2f8fa82c94e0815 WatchSource:0}: Error finding container cde99ac77f9fbcc2167df80afb15cb4423e7cfb4e0708c81b2f8fa82c94e0815: Status 404 returned error can't find the container with id cde99ac77f9fbcc2167df80afb15cb4423e7cfb4e0708c81b2f8fa82c94e0815 Aug 04 12:29:45.770472 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:45.770407 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d"] Aug 04 12:29:45.781427 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:29:45.781400 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddfb6594_e3fd_487a_9556_f1c779926ec0.slice/crio-9b915c58f9e9d45840f577319bd0ab9cc9fc338030ffad460579c5b8e0115874 WatchSource:0}: Error finding container 9b915c58f9e9d45840f577319bd0ab9cc9fc338030ffad460579c5b8e0115874: Status 404 returned error can't find the container with id 9b915c58f9e9d45840f577319bd0ab9cc9fc338030ffad460579c5b8e0115874 Aug 04 12:29:46.001753 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:46.001717 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d" event={"ID":"ddfb6594-e3fd-487a-9556-f1c779926ec0","Type":"ContainerStarted","Data":"4c5bc762f2663390db7a822e8629cd809906fbb2d59e056eec60501072552ee7"} Aug 04 12:29:46.001931 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:46.001775 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d" event={"ID":"ddfb6594-e3fd-487a-9556-f1c779926ec0","Type":"ContainerStarted","Data":"9b915c58f9e9d45840f577319bd0ab9cc9fc338030ffad460579c5b8e0115874"} Aug 04 12:29:46.001931 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:46.001837 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d" Aug 04 12:29:46.003173 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:46.003149 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h" event={"ID":"fd830dcf-41a3-423d-8e54-6d9438ee0db9","Type":"ContainerStarted","Data":"a1b02f06c6ef534061755aaf70748d55072593ef6d9014651c5f1407ede16336"} Aug 04 12:29:46.003306 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:46.003194 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h" event={"ID":"fd830dcf-41a3-423d-8e54-6d9438ee0db9","Type":"ContainerStarted","Data":"cde99ac77f9fbcc2167df80afb15cb4423e7cfb4e0708c81b2f8fa82c94e0815"} Aug 04 12:29:46.003306 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:46.003221 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h" Aug 04 12:29:46.024817 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:46.024771 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d" podStartSLOduration=1.024758734 podStartE2EDuration="1.024758734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:29:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:29:46.022959526 +0000 UTC m=+3398.267679067" watchObservedRunningTime="2026-08-04 12:29:46.024758734 +0000 UTC m=+3398.269478277" Aug 04 12:29:46.062208 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:46.062125 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h" podStartSLOduration=1.062107532 podStartE2EDuration="1.062107532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:29:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:29:46.061235133 +0000 UTC m=+3398.305954944" watchObservedRunningTime="2026-08-04 12:29:46.062107532 +0000 UTC m=+3398.306827078" Aug 04 12:29:57.009727 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:57.009697 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h" Aug 04 12:29:57.010273 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:29:57.009888 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d" Aug 04 12:30:00.149879 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:00.149840 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a888a4-29764110-cfdpk"] Aug 04 12:30:00.153238 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:00.153213 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764110-wzpv8"] Aug 04 12:30:00.153383 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:00.153325 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a888a4-29764110-cfdpk" Aug 04 12:30:00.156105 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:00.156081 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pv6cr\"" Aug 04 12:30:00.156304 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:00.156290 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-038ca6-29764110-4jm9f"] Aug 04 12:30:00.156429 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:00.156413 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764110-wzpv8" Aug 04 12:30:00.159285 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:00.159268 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7013e-29764110-ngtbf"] Aug 04 12:30:00.159410 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:00.159391 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-038ca6-29764110-4jm9f" Aug 04 12:30:00.164134 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:00.164111 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-038ca6-29764110-4jm9f"] Aug 04 12:30:00.164254 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:00.164202 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7013e-29764110-ngtbf" Aug 04 12:30:00.165557 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:00.165535 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764110-wzpv8"] Aug 04 12:30:00.166938 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:00.166747 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a888a4-29764110-cfdpk" Aug 04 12:30:00.169528 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:00.169004 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7013e-29764110-ngtbf"] Aug 04 12:30:00.171229 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:00.171203 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a888a4-29764110-cfdpk"] Aug 04 12:30:00.177526 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:00.177508 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764110-wzpv8" Aug 04 12:30:00.208601 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:00.208579 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-038ca6-29764110-4jm9f" Aug 04 12:30:00.215946 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:00.215796 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7013e-29764110-ngtbf" Aug 04 12:30:00.360235 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:00.360141 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a888a4-29764110-cfdpk"] Aug 04 12:30:00.360935 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:30:00.360904 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92f5134d_346b_416b_b3f5_a6706f816468.slice/crio-2ec2f8f1de4cfae5a8e8535745e26e6422fdfa2349aa2bb67d2b92c3a5992e0e WatchSource:0}: Error finding container 2ec2f8f1de4cfae5a8e8535745e26e6422fdfa2349aa2bb67d2b92c3a5992e0e: Status 404 returned error can't find the container with id 2ec2f8f1de4cfae5a8e8535745e26e6422fdfa2349aa2bb67d2b92c3a5992e0e Aug 04 12:30:00.591750 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:00.591609 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764110-wzpv8"] Aug 04 12:30:00.594222 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:30:00.594199 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8b25435_ae90_48e1_949d_fce97991cf99.slice/crio-d0ab8ac5d89ac2ae37e1ee8cba034075ca9cc15b17002ee73cc344665a275995 WatchSource:0}: Error finding container d0ab8ac5d89ac2ae37e1ee8cba034075ca9cc15b17002ee73cc344665a275995: Status 404 returned error can't find the container with id d0ab8ac5d89ac2ae37e1ee8cba034075ca9cc15b17002ee73cc344665a275995 Aug 04 12:30:00.614888 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:00.614869 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-038ca6-29764110-4jm9f"] Aug 04 12:30:00.624000 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:00.623980 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7013e-29764110-ngtbf"] Aug 04 12:30:00.625658 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:30:00.625633 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07da324e_3515_4528_bf85_29930ce23f9b.slice/crio-f44799a43b8e7b1efa01230b9db6bdebbcbd81acd89af0f94687aa548a6d7f0d WatchSource:0}: Error finding container f44799a43b8e7b1efa01230b9db6bdebbcbd81acd89af0f94687aa548a6d7f0d: Status 404 returned error can't find the container with id f44799a43b8e7b1efa01230b9db6bdebbcbd81acd89af0f94687aa548a6d7f0d Aug 04 12:30:00.625858 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:30:00.625828 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c63924b_cb3b_45e9_826d_614cd9fe81ae.slice/crio-3ba0241be0f4d5becf53805a610377be3add0030e5ec21123901661945d92f4a WatchSource:0}: Error finding container 3ba0241be0f4d5becf53805a610377be3add0030e5ec21123901661945d92f4a: Status 404 returned error can't find the container with id 3ba0241be0f4d5becf53805a610377be3add0030e5ec21123901661945d92f4a Aug 04 12:30:01.072314 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:01.072281 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7013e-29764110-ngtbf" event={"ID":"8c63924b-cb3b-45e9-826d-614cd9fe81ae","Type":"ContainerStarted","Data":"aafc41b94bba9589ba2174f24f9b6440939a6147f1685a0a927b16ec2a95c8e9"} Aug 04 12:30:01.072475 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:01.072341 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7013e-29764110-ngtbf" event={"ID":"8c63924b-cb3b-45e9-826d-614cd9fe81ae","Type":"ContainerStarted","Data":"3ba0241be0f4d5becf53805a610377be3add0030e5ec21123901661945d92f4a"} Aug 04 12:30:01.073748 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:01.073722 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a888a4-29764110-cfdpk" event={"ID":"92f5134d-346b-416b-b3f5-a6706f816468","Type":"ContainerStarted","Data":"03be186e8406d189d77b8890c68ab4396314163c5e5d644a15dedbcddefdd8f6"} Aug 04 12:30:01.073872 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:01.073764 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a888a4-29764110-cfdpk" event={"ID":"92f5134d-346b-416b-b3f5-a6706f816468","Type":"ContainerStarted","Data":"2ec2f8f1de4cfae5a8e8535745e26e6422fdfa2349aa2bb67d2b92c3a5992e0e"} Aug 04 12:30:01.075077 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:01.075046 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764110-wzpv8" event={"ID":"a8b25435-ae90-48e1-949d-fce97991cf99","Type":"ContainerStarted","Data":"c02b3dc889707f862c028f0558cc6404d775bea6f5fa22a7b33843c196ae7856"} Aug 04 12:30:01.075175 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:01.075090 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764110-wzpv8" event={"ID":"a8b25435-ae90-48e1-949d-fce97991cf99","Type":"ContainerStarted","Data":"d0ab8ac5d89ac2ae37e1ee8cba034075ca9cc15b17002ee73cc344665a275995"} Aug 04 12:30:01.076381 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:01.076358 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-038ca6-29764110-4jm9f" event={"ID":"07da324e-3515-4528-bf85-29930ce23f9b","Type":"ContainerStarted","Data":"220775df09ddc6be6b2e6109232636bd127a1d3ab3c7003a1456198ee6254660"} Aug 04 12:30:01.076501 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:01.076402 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-038ca6-29764110-4jm9f" event={"ID":"07da324e-3515-4528-bf85-29930ce23f9b","Type":"ContainerStarted","Data":"f44799a43b8e7b1efa01230b9db6bdebbcbd81acd89af0f94687aa548a6d7f0d"} Aug 04 12:30:01.090409 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:01.090370 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7013e-29764110-ngtbf" podStartSLOduration=1.09035462 podStartE2EDuration="1.09035462s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:30:01.089223467 +0000 UTC m=+3413.333943019" watchObservedRunningTime="2026-08-04 12:30:01.09035462 +0000 UTC m=+3413.335074143" Aug 04 12:30:01.106539 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:01.106471 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a888a4-29764110-cfdpk" podStartSLOduration=1.10644105 podStartE2EDuration="1.10644105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:30:01.105358296 +0000 UTC m=+3413.350077839" watchObservedRunningTime="2026-08-04 12:30:01.10644105 +0000 UTC m=+3413.351160592" Aug 04 12:30:01.124836 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:01.124802 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764110-wzpv8" podStartSLOduration=1.124791157 podStartE2EDuration="1.124791157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:30:01.124369597 +0000 UTC m=+3413.369089139" watchObservedRunningTime="2026-08-04 12:30:01.124791157 +0000 UTC m=+3413.369510698" Aug 04 12:30:01.146699 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:01.146663 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-038ca6-29764110-4jm9f" podStartSLOduration=1.146652409 podStartE2EDuration="1.146652409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:30:01.14470537 +0000 UTC m=+3413.389424911" watchObservedRunningTime="2026-08-04 12:30:01.146652409 +0000 UTC m=+3413.391371952" Aug 04 12:30:02.087883 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:02.087846 2570 generic.go:356] "Generic (PLEG): container finished" podID="8c63924b-cb3b-45e9-826d-614cd9fe81ae" containerID="aafc41b94bba9589ba2174f24f9b6440939a6147f1685a0a927b16ec2a95c8e9" exitCode=0 Aug 04 12:30:02.088299 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:02.087926 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7013e-29764110-ngtbf" event={"ID":"8c63924b-cb3b-45e9-826d-614cd9fe81ae","Type":"ContainerDied","Data":"aafc41b94bba9589ba2174f24f9b6440939a6147f1685a0a927b16ec2a95c8e9"} Aug 04 12:30:02.089381 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:02.089358 2570 generic.go:356] "Generic (PLEG): container finished" podID="92f5134d-346b-416b-b3f5-a6706f816468" containerID="03be186e8406d189d77b8890c68ab4396314163c5e5d644a15dedbcddefdd8f6" exitCode=0 Aug 04 12:30:02.089480 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:02.089440 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a888a4-29764110-cfdpk" event={"ID":"92f5134d-346b-416b-b3f5-a6706f816468","Type":"ContainerDied","Data":"03be186e8406d189d77b8890c68ab4396314163c5e5d644a15dedbcddefdd8f6"} Aug 04 12:30:02.090768 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:02.090745 2570 generic.go:356] "Generic (PLEG): container finished" podID="a8b25435-ae90-48e1-949d-fce97991cf99" containerID="c02b3dc889707f862c028f0558cc6404d775bea6f5fa22a7b33843c196ae7856" exitCode=0 Aug 04 12:30:02.090877 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:02.090827 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764110-wzpv8" event={"ID":"a8b25435-ae90-48e1-949d-fce97991cf99","Type":"ContainerDied","Data":"c02b3dc889707f862c028f0558cc6404d775bea6f5fa22a7b33843c196ae7856"} Aug 04 12:30:02.092105 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:02.092085 2570 generic.go:356] "Generic (PLEG): container finished" podID="07da324e-3515-4528-bf85-29930ce23f9b" containerID="220775df09ddc6be6b2e6109232636bd127a1d3ab3c7003a1456198ee6254660" exitCode=0 Aug 04 12:30:02.092186 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:02.092109 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-038ca6-29764110-4jm9f" event={"ID":"07da324e-3515-4528-bf85-29930ce23f9b","Type":"ContainerDied","Data":"220775df09ddc6be6b2e6109232636bd127a1d3ab3c7003a1456198ee6254660"} Aug 04 12:30:03.245494 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:03.245467 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764110-wzpv8" Aug 04 12:30:03.278987 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:03.278906 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a888a4-29764110-cfdpk" Aug 04 12:30:03.315320 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:03.315300 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-038ca6-29764110-4jm9f" Aug 04 12:30:03.318500 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:03.318483 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7013e-29764110-ngtbf" Aug 04 12:30:04.102255 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:04.102222 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764110-wzpv8" Aug 04 12:30:04.102463 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:04.102253 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764110-wzpv8" event={"ID":"a8b25435-ae90-48e1-949d-fce97991cf99","Type":"ContainerDied","Data":"d0ab8ac5d89ac2ae37e1ee8cba034075ca9cc15b17002ee73cc344665a275995"} Aug 04 12:30:04.102463 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:04.102304 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0ab8ac5d89ac2ae37e1ee8cba034075ca9cc15b17002ee73cc344665a275995" Aug 04 12:30:04.103508 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:04.103481 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-038ca6-29764110-4jm9f" event={"ID":"07da324e-3515-4528-bf85-29930ce23f9b","Type":"ContainerDied","Data":"f44799a43b8e7b1efa01230b9db6bdebbcbd81acd89af0f94687aa548a6d7f0d"} Aug 04 12:30:04.103508 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:04.103502 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-038ca6-29764110-4jm9f" Aug 04 12:30:04.103683 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:04.103529 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f44799a43b8e7b1efa01230b9db6bdebbcbd81acd89af0f94687aa548a6d7f0d" Aug 04 12:30:04.104911 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:04.104895 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7013e-29764110-ngtbf" Aug 04 12:30:04.105021 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:04.104929 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7013e-29764110-ngtbf" event={"ID":"8c63924b-cb3b-45e9-826d-614cd9fe81ae","Type":"ContainerDied","Data":"3ba0241be0f4d5becf53805a610377be3add0030e5ec21123901661945d92f4a"} Aug 04 12:30:04.105021 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:04.104966 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ba0241be0f4d5becf53805a610377be3add0030e5ec21123901661945d92f4a" Aug 04 12:30:04.106307 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:04.106289 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a888a4-29764110-cfdpk" event={"ID":"92f5134d-346b-416b-b3f5-a6706f816468","Type":"ContainerDied","Data":"2ec2f8f1de4cfae5a8e8535745e26e6422fdfa2349aa2bb67d2b92c3a5992e0e"} Aug 04 12:30:04.106398 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:04.106306 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a888a4-29764110-cfdpk" Aug 04 12:30:04.106398 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:04.106326 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ec2f8f1de4cfae5a8e8535745e26e6422fdfa2349aa2bb67d2b92c3a5992e0e" Aug 04 12:30:14.473554 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.473510 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb"] Aug 04 12:30:14.474137 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.474089 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c63924b-cb3b-45e9-826d-614cd9fe81ae" containerName="cleanup" Aug 04 12:30:14.474137 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.474111 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c63924b-cb3b-45e9-826d-614cd9fe81ae" containerName="cleanup" Aug 04 12:30:14.474137 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.474123 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92f5134d-346b-416b-b3f5-a6706f816468" containerName="cleanup" Aug 04 12:30:14.474137 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.474131 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="92f5134d-346b-416b-b3f5-a6706f816468" containerName="cleanup" Aug 04 12:30:14.474352 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.474145 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8b25435-ae90-48e1-949d-fce97991cf99" containerName="cleanup" Aug 04 12:30:14.474352 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.474154 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8b25435-ae90-48e1-949d-fce97991cf99" containerName="cleanup" Aug 04 12:30:14.474352 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.474166 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07da324e-3515-4528-bf85-29930ce23f9b" containerName="cleanup" Aug 04 12:30:14.474352 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.474174 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="07da324e-3515-4528-bf85-29930ce23f9b" containerName="cleanup" Aug 04 12:30:14.474352 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.474255 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="92f5134d-346b-416b-b3f5-a6706f816468" containerName="cleanup" Aug 04 12:30:14.474352 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.474274 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="07da324e-3515-4528-bf85-29930ce23f9b" containerName="cleanup" Aug 04 12:30:14.474352 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.474284 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8b25435-ae90-48e1-949d-fce97991cf99" containerName="cleanup" Aug 04 12:30:14.474352 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.474293 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c63924b-cb3b-45e9-826d-614cd9fe81ae" containerName="cleanup" Aug 04 12:30:14.477793 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.477768 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb" Aug 04 12:30:14.479236 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.479211 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf"] Aug 04 12:30:14.482427 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.482411 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf" Aug 04 12:30:14.483150 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.483135 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-0778c1\"" Aug 04 12:30:14.597678 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.597634 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2jng\" (UniqueName: \"kubernetes.io/projected/808db485-b189-4cbe-8a89-767d553ba7cf-kube-api-access-c2jng\") pod \"payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb\" (UID: \"808db485-b189-4cbe-8a89-767d553ba7cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb" Aug 04 12:30:14.597861 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.597704 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce40eea1-0ec5-4c56-8af4-0a9f4480266f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf\" (UID: \"ce40eea1-0ec5-4c56-8af4-0a9f4480266f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf" Aug 04 12:30:14.597861 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.597789 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/808db485-b189-4cbe-8a89-767d553ba7cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb\" (UID: \"808db485-b189-4cbe-8a89-767d553ba7cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb" Aug 04 12:30:14.597861 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.597825 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mnnr\" (UniqueName: \"kubernetes.io/projected/ce40eea1-0ec5-4c56-8af4-0a9f4480266f-kube-api-access-9mnnr\") pod \"payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf\" (UID: \"ce40eea1-0ec5-4c56-8af4-0a9f4480266f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf" Aug 04 12:30:14.698662 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.698626 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/808db485-b189-4cbe-8a89-767d553ba7cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb\" (UID: \"808db485-b189-4cbe-8a89-767d553ba7cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb" Aug 04 12:30:14.698662 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.698668 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mnnr\" (UniqueName: \"kubernetes.io/projected/ce40eea1-0ec5-4c56-8af4-0a9f4480266f-kube-api-access-9mnnr\") pod \"payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf\" (UID: \"ce40eea1-0ec5-4c56-8af4-0a9f4480266f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf" Aug 04 12:30:14.698886 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.698712 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2jng\" (UniqueName: \"kubernetes.io/projected/808db485-b189-4cbe-8a89-767d553ba7cf-kube-api-access-c2jng\") pod \"payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb\" (UID: \"808db485-b189-4cbe-8a89-767d553ba7cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb" Aug 04 12:30:14.698886 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.698742 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce40eea1-0ec5-4c56-8af4-0a9f4480266f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf\" (UID: \"ce40eea1-0ec5-4c56-8af4-0a9f4480266f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf" Aug 04 12:30:14.699381 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.699363 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce40eea1-0ec5-4c56-8af4-0a9f4480266f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf\" (UID: \"ce40eea1-0ec5-4c56-8af4-0a9f4480266f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf" Aug 04 12:30:14.699434 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.699414 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/808db485-b189-4cbe-8a89-767d553ba7cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb\" (UID: \"808db485-b189-4cbe-8a89-767d553ba7cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb" Aug 04 12:30:14.708258 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.708230 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mnnr\" (UniqueName: \"kubernetes.io/projected/ce40eea1-0ec5-4c56-8af4-0a9f4480266f-kube-api-access-9mnnr\") pod \"payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf\" (UID: \"ce40eea1-0ec5-4c56-8af4-0a9f4480266f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf" Aug 04 12:30:14.708377 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.708332 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2jng\" (UniqueName: \"kubernetes.io/projected/808db485-b189-4cbe-8a89-767d553ba7cf-kube-api-access-c2jng\") pod \"payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb\" (UID: \"808db485-b189-4cbe-8a89-767d553ba7cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb" Aug 04 12:30:14.789514 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.789424 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb" Aug 04 12:30:14.795613 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.795582 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf" Aug 04 12:30:14.892355 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.892108 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb"] Aug 04 12:30:14.894034 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:14.894008 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf"] Aug 04 12:30:15.154880 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:15.154854 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb"] Aug 04 12:30:15.158076 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:30:15.158046 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod808db485_b189_4cbe_8a89_767d553ba7cf.slice/crio-9b79906dbc882177f101b955f3e1e1cc359267c1167c3f6f7cd61aea3ad1bc78 WatchSource:0}: Error finding container 9b79906dbc882177f101b955f3e1e1cc359267c1167c3f6f7cd61aea3ad1bc78: Status 404 returned error can't find the container with id 9b79906dbc882177f101b955f3e1e1cc359267c1167c3f6f7cd61aea3ad1bc78 Aug 04 12:30:15.181711 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:15.181683 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf"] Aug 04 12:30:15.189747 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:30:15.189649 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce40eea1_0ec5_4c56_8af4_0a9f4480266f.slice/crio-a08bd8d6ca282892dd07a6a0aedf15e644a91832474085da00ab732c80009942 WatchSource:0}: Error finding container a08bd8d6ca282892dd07a6a0aedf15e644a91832474085da00ab732c80009942: Status 404 returned error can't find the container with id a08bd8d6ca282892dd07a6a0aedf15e644a91832474085da00ab732c80009942 Aug 04 12:30:16.164560 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:16.164517 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf" event={"ID":"ce40eea1-0ec5-4c56-8af4-0a9f4480266f","Type":"ContainerStarted","Data":"922d247a2c271b5f50e8ea4096f08b4fb5efcba4d89e88402039f2710161f53c"} Aug 04 12:30:16.165063 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:16.164624 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf" Aug 04 12:30:16.165063 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:16.164649 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf" event={"ID":"ce40eea1-0ec5-4c56-8af4-0a9f4480266f","Type":"ContainerStarted","Data":"a08bd8d6ca282892dd07a6a0aedf15e644a91832474085da00ab732c80009942"} Aug 04 12:30:16.166272 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:16.166247 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb" event={"ID":"808db485-b189-4cbe-8a89-767d553ba7cf","Type":"ContainerStarted","Data":"714fa479b42ef6604a989aada220e3b1e2e00769b926ed4631cf96ee6516a80e"} Aug 04 12:30:16.166373 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:16.166292 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb" event={"ID":"808db485-b189-4cbe-8a89-767d553ba7cf","Type":"ContainerStarted","Data":"9b79906dbc882177f101b955f3e1e1cc359267c1167c3f6f7cd61aea3ad1bc78"} Aug 04 12:30:16.166373 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:16.166318 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb" Aug 04 12:30:16.182566 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:16.182510 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf" podStartSLOduration=2.18249615 podStartE2EDuration="2.18249615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:30:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:30:16.181424223 +0000 UTC m=+3428.426143791" watchObservedRunningTime="2026-08-04 12:30:16.18249615 +0000 UTC m=+3428.427215670" Aug 04 12:30:16.200701 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:16.200652 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb" podStartSLOduration=2.200634764 podStartE2EDuration="2.200634764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:30:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:30:16.200037739 +0000 UTC m=+3428.444757282" watchObservedRunningTime="2026-08-04 12:30:16.200634764 +0000 UTC m=+3428.445354307" Aug 04 12:30:27.174635 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:27.174587 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf" Aug 04 12:30:27.175057 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:27.174849 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb" Aug 04 12:30:45.054563 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:30:45.054528 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf_ce40eea1-0ec5-4c56-8af4-0a9f4480266f/payload-processing/0.log" Aug 04 12:32:00.529082 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:00.529007 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d_ddfb6594-e3fd-487a-9556-f1c779926ec0/payload-processing/0.log" Aug 04 12:32:00.761658 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:00.761632 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf_ce40eea1-0ec5-4c56-8af4-0a9f4480266f/payload-processing/0.log" Aug 04 12:32:05.815941 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:05.815906 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6"] Aug 04 12:32:05.819401 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:05.819384 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6" Aug 04 12:32:05.823061 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:05.823039 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-672896\"" Aug 04 12:32:05.823189 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:05.823042 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-672896-dockercfg-f8zxd\"" Aug 04 12:32:05.831549 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:05.831524 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6"] Aug 04 12:32:05.839535 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:05.839507 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756"] Aug 04 12:32:05.842965 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:05.842944 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756" Aug 04 12:32:05.854574 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:05.854537 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756"] Aug 04 12:32:05.904689 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:05.899414 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z989f\" (UniqueName: \"kubernetes.io/projected/f79ddf8e-5d4f-4449-bebc-5cba12c47f4e-kube-api-access-z989f\") pod \"payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6\" (UID: \"f79ddf8e-5d4f-4449-bebc-5cba12c47f4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6" Aug 04 12:32:05.904689 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:05.899564 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f79ddf8e-5d4f-4449-bebc-5cba12c47f4e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6\" (UID: \"f79ddf8e-5d4f-4449-bebc-5cba12c47f4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6" Aug 04 12:32:06.001028 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:06.000984 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gbft\" (UniqueName: \"kubernetes.io/projected/a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8-kube-api-access-4gbft\") pod \"payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756\" (UID: \"a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756" Aug 04 12:32:06.001028 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:06.001036 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f79ddf8e-5d4f-4449-bebc-5cba12c47f4e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6\" (UID: \"f79ddf8e-5d4f-4449-bebc-5cba12c47f4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6" Aug 04 12:32:06.001263 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:06.001127 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z989f\" (UniqueName: \"kubernetes.io/projected/f79ddf8e-5d4f-4449-bebc-5cba12c47f4e-kube-api-access-z989f\") pod \"payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6\" (UID: \"f79ddf8e-5d4f-4449-bebc-5cba12c47f4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6" Aug 04 12:32:06.001263 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:06.001200 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756\" (UID: \"a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756" Aug 04 12:32:06.001785 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:06.001767 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f79ddf8e-5d4f-4449-bebc-5cba12c47f4e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6\" (UID: \"f79ddf8e-5d4f-4449-bebc-5cba12c47f4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6" Aug 04 12:32:06.010761 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:06.010737 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z989f\" (UniqueName: \"kubernetes.io/projected/f79ddf8e-5d4f-4449-bebc-5cba12c47f4e-kube-api-access-z989f\") pod \"payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6\" (UID: \"f79ddf8e-5d4f-4449-bebc-5cba12c47f4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6" Aug 04 12:32:06.101905 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:06.101805 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756\" (UID: \"a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756" Aug 04 12:32:06.101905 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:06.101875 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gbft\" (UniqueName: \"kubernetes.io/projected/a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8-kube-api-access-4gbft\") pod \"payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756\" (UID: \"a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756" Aug 04 12:32:06.102587 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:06.102566 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756\" (UID: \"a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756" Aug 04 12:32:06.110475 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:06.110427 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gbft\" (UniqueName: \"kubernetes.io/projected/a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8-kube-api-access-4gbft\") pod \"payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756\" (UID: \"a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756" Aug 04 12:32:06.130314 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:06.130281 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6" Aug 04 12:32:06.155443 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:06.155409 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756" Aug 04 12:32:06.285885 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:06.285851 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6"] Aug 04 12:32:06.287784 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:32:06.287748 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf79ddf8e_5d4f_4449_bebc_5cba12c47f4e.slice/crio-a7941dbd24f4ec9477e4d9dd157d049bac58c10aafa4d51abb539c261a322dde WatchSource:0}: Error finding container a7941dbd24f4ec9477e4d9dd157d049bac58c10aafa4d51abb539c261a322dde: Status 404 returned error can't find the container with id a7941dbd24f4ec9477e4d9dd157d049bac58c10aafa4d51abb539c261a322dde Aug 04 12:32:06.313085 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:06.313051 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756"] Aug 04 12:32:06.321666 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:32:06.321634 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda04255c3_0bc9_4a2b_a81c_6cbd6e460fe8.slice/crio-f20d5f708414902f86319c19298d01d7161f2025af91a5c4d661cb202690735a WatchSource:0}: Error finding container f20d5f708414902f86319c19298d01d7161f2025af91a5c4d661cb202690735a: Status 404 returned error can't find the container with id f20d5f708414902f86319c19298d01d7161f2025af91a5c4d661cb202690735a Aug 04 12:32:06.691146 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:06.691051 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6" event={"ID":"f79ddf8e-5d4f-4449-bebc-5cba12c47f4e","Type":"ContainerStarted","Data":"466ef870ec0f712eccaef4ec9243d8d944e5e34cdfd646447f73219aff2ee84a"} Aug 04 12:32:06.691146 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:06.691113 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6" event={"ID":"f79ddf8e-5d4f-4449-bebc-5cba12c47f4e","Type":"ContainerStarted","Data":"a7941dbd24f4ec9477e4d9dd157d049bac58c10aafa4d51abb539c261a322dde"} Aug 04 12:32:06.691377 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:06.691229 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6" Aug 04 12:32:06.692527 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:06.692505 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756" event={"ID":"a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8","Type":"ContainerStarted","Data":"2e0aeeaac89cd4d521567628f101bed9faef4e73be92dd1aa2abc5e29f125979"} Aug 04 12:32:06.692593 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:06.692551 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756" event={"ID":"a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8","Type":"ContainerStarted","Data":"f20d5f708414902f86319c19298d01d7161f2025af91a5c4d661cb202690735a"} Aug 04 12:32:06.692593 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:06.692583 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756" Aug 04 12:32:06.708789 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:06.708740 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6" podStartSLOduration=1.7087242470000001 podStartE2EDuration="1.708724247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:32:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:32:06.707845689 +0000 UTC m=+3538.952565258" watchObservedRunningTime="2026-08-04 12:32:06.708724247 +0000 UTC m=+3538.953443790" Aug 04 12:32:06.732215 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:06.732174 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756" podStartSLOduration=1.732160685 podStartE2EDuration="1.732160685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:32:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:32:06.730662484 +0000 UTC m=+3538.975382052" watchObservedRunningTime="2026-08-04 12:32:06.732160685 +0000 UTC m=+3538.976880279" Aug 04 12:32:17.699194 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:17.699158 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756" Aug 04 12:32:17.699612 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:17.699396 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6" Aug 04 12:32:29.704653 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:29.704620 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-672896-sttk5"] Aug 04 12:32:29.708120 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:29.708095 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-672896-sttk5" Aug 04 12:32:29.711478 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:29.711461 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pv6cr\"" Aug 04 12:32:29.725466 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:29.725404 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-672896-sttk5"] Aug 04 12:32:29.825516 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:29.825481 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/222451ee-b365-4f39-90f3-8f93e88a733c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-672896-sttk5\" (UID: \"222451ee-b365-4f39-90f3-8f93e88a733c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-672896-sttk5" Aug 04 12:32:29.926912 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:29.926877 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/222451ee-b365-4f39-90f3-8f93e88a733c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-672896-sttk5\" (UID: \"222451ee-b365-4f39-90f3-8f93e88a733c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-672896-sttk5" Aug 04 12:32:29.927543 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:29.927524 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/222451ee-b365-4f39-90f3-8f93e88a733c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-672896-sttk5\" (UID: \"222451ee-b365-4f39-90f3-8f93e88a733c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-672896-sttk5" Aug 04 12:32:30.026621 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:30.026587 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-672896-sttk5" Aug 04 12:32:30.160459 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:30.160421 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-672896-sttk5"] Aug 04 12:32:30.162767 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:32:30.162738 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod222451ee_b365_4f39_90f3_8f93e88a733c.slice/crio-460090af70e6f807804a9ca95de51508376eb2b03baba9b9eb79121e820f070c WatchSource:0}: Error finding container 460090af70e6f807804a9ca95de51508376eb2b03baba9b9eb79121e820f070c: Status 404 returned error can't find the container with id 460090af70e6f807804a9ca95de51508376eb2b03baba9b9eb79121e820f070c Aug 04 12:32:30.812510 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:30.812475 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-672896-sttk5" event={"ID":"222451ee-b365-4f39-90f3-8f93e88a733c","Type":"ContainerStarted","Data":"8b30d331814ff7eb3c512e925994b17fdb9c18328d56f69d961a55687d628e73"} Aug 04 12:32:30.812937 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:30.812530 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-672896-sttk5" event={"ID":"222451ee-b365-4f39-90f3-8f93e88a733c","Type":"ContainerStarted","Data":"460090af70e6f807804a9ca95de51508376eb2b03baba9b9eb79121e820f070c"} Aug 04 12:32:30.834193 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:30.834154 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-672896-sttk5" podStartSLOduration=1.83413862 podStartE2EDuration="1.83413862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:32:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:32:30.832130357 +0000 UTC m=+3563.076849900" watchObservedRunningTime="2026-08-04 12:32:30.83413862 +0000 UTC m=+3563.078858162" Aug 04 12:32:31.818811 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:31.818779 2570 generic.go:356] "Generic (PLEG): container finished" podID="222451ee-b365-4f39-90f3-8f93e88a733c" containerID="8b30d331814ff7eb3c512e925994b17fdb9c18328d56f69d961a55687d628e73" exitCode=0 Aug 04 12:32:31.819265 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:31.818852 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-672896-sttk5" event={"ID":"222451ee-b365-4f39-90f3-8f93e88a733c","Type":"ContainerDied","Data":"8b30d331814ff7eb3c512e925994b17fdb9c18328d56f69d961a55687d628e73"} Aug 04 12:32:32.954750 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:32.954724 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-672896-sttk5" Aug 04 12:32:33.052325 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:33.052289 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/222451ee-b365-4f39-90f3-8f93e88a733c-maas-api-service-ca\") pod \"222451ee-b365-4f39-90f3-8f93e88a733c\" (UID: \"222451ee-b365-4f39-90f3-8f93e88a733c\") " Aug 04 12:32:33.052679 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:33.052655 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/222451ee-b365-4f39-90f3-8f93e88a733c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "222451ee-b365-4f39-90f3-8f93e88a733c" (UID: "222451ee-b365-4f39-90f3-8f93e88a733c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:32:33.153534 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:33.153464 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/222451ee-b365-4f39-90f3-8f93e88a733c-maas-api-service-ca\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:32:33.828963 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:33.828930 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-672896-sttk5" event={"ID":"222451ee-b365-4f39-90f3-8f93e88a733c","Type":"ContainerDied","Data":"460090af70e6f807804a9ca95de51508376eb2b03baba9b9eb79121e820f070c"} Aug 04 12:32:33.828963 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:33.828961 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-672896-sttk5" Aug 04 12:32:33.829172 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:33.828979 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="460090af70e6f807804a9ca95de51508376eb2b03baba9b9eb79121e820f070c" Aug 04 12:32:35.999476 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:35.999428 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756"] Aug 04 12:32:35.999845 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:35.999672 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756" podUID="a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8" containerName="payload-processing" containerID="cri-o://2e0aeeaac89cd4d521567628f101bed9faef4e73be92dd1aa2abc5e29f125979" gracePeriod=30 Aug 04 12:32:36.007177 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.006720 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6"] Aug 04 12:32:36.008785 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.008751 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6" podUID="f79ddf8e-5d4f-4449-bebc-5cba12c47f4e" containerName="payload-pre-processing" containerID="cri-o://466ef870ec0f712eccaef4ec9243d8d944e5e34cdfd646447f73219aff2ee84a" gracePeriod=30 Aug 04 12:32:36.181471 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.181415 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-672896-sttk5"] Aug 04 12:32:36.181753 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.181736 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-672896-sttk5"] Aug 04 12:32:36.198430 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.196612 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="222451ee-b365-4f39-90f3-8f93e88a733c" path="/var/lib/kubelet/pods/222451ee-b365-4f39-90f3-8f93e88a733c/volumes" Aug 04 12:32:36.288551 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.288525 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756" Aug 04 12:32:36.320799 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.320776 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6" Aug 04 12:32:36.382753 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.382724 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z989f\" (UniqueName: \"kubernetes.io/projected/f79ddf8e-5d4f-4449-bebc-5cba12c47f4e-kube-api-access-z989f\") pod \"f79ddf8e-5d4f-4449-bebc-5cba12c47f4e\" (UID: \"f79ddf8e-5d4f-4449-bebc-5cba12c47f4e\") " Aug 04 12:32:36.382892 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.382768 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f79ddf8e-5d4f-4449-bebc-5cba12c47f4e-plugins-config-volume\") pod \"f79ddf8e-5d4f-4449-bebc-5cba12c47f4e\" (UID: \"f79ddf8e-5d4f-4449-bebc-5cba12c47f4e\") " Aug 04 12:32:36.382968 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.382948 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gbft\" (UniqueName: \"kubernetes.io/projected/a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8-kube-api-access-4gbft\") pod \"a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8\" (UID: \"a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8\") " Aug 04 12:32:36.383097 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.383075 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8-plugins-config-volume\") pod \"a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8\" (UID: \"a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8\") " Aug 04 12:32:36.383219 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.383123 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f79ddf8e-5d4f-4449-bebc-5cba12c47f4e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f79ddf8e-5d4f-4449-bebc-5cba12c47f4e" (UID: "f79ddf8e-5d4f-4449-bebc-5cba12c47f4e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:32:36.383376 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.383355 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f79ddf8e-5d4f-4449-bebc-5cba12c47f4e-plugins-config-volume\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:32:36.383484 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.383379 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8" (UID: "a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:32:36.384809 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.384792 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f79ddf8e-5d4f-4449-bebc-5cba12c47f4e-kube-api-access-z989f" (OuterVolumeSpecName: "kube-api-access-z989f") pod "f79ddf8e-5d4f-4449-bebc-5cba12c47f4e" (UID: "f79ddf8e-5d4f-4449-bebc-5cba12c47f4e"). InnerVolumeSpecName "kube-api-access-z989f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:32:36.384924 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.384905 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8-kube-api-access-4gbft" (OuterVolumeSpecName: "kube-api-access-4gbft") pod "a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8" (UID: "a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8"). InnerVolumeSpecName "kube-api-access-4gbft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:32:36.484711 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.484686 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8-plugins-config-volume\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:32:36.484711 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.484708 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z989f\" (UniqueName: \"kubernetes.io/projected/f79ddf8e-5d4f-4449-bebc-5cba12c47f4e-kube-api-access-z989f\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:32:36.484711 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.484717 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gbft\" (UniqueName: \"kubernetes.io/projected/a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8-kube-api-access-4gbft\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:32:36.844569 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.844528 2570 generic.go:356] "Generic (PLEG): container finished" podID="a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8" containerID="2e0aeeaac89cd4d521567628f101bed9faef4e73be92dd1aa2abc5e29f125979" exitCode=0 Aug 04 12:32:36.844736 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.844576 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756" Aug 04 12:32:36.844736 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.844623 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756" event={"ID":"a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8","Type":"ContainerDied","Data":"2e0aeeaac89cd4d521567628f101bed9faef4e73be92dd1aa2abc5e29f125979"} Aug 04 12:32:36.844736 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.844672 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756" event={"ID":"a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8","Type":"ContainerDied","Data":"f20d5f708414902f86319c19298d01d7161f2025af91a5c4d661cb202690735a"} Aug 04 12:32:36.844736 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.844702 2570 scope.go:117] "RemoveContainer" containerID="2e0aeeaac89cd4d521567628f101bed9faef4e73be92dd1aa2abc5e29f125979" Aug 04 12:32:36.846119 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.846094 2570 generic.go:356] "Generic (PLEG): container finished" podID="f79ddf8e-5d4f-4449-bebc-5cba12c47f4e" containerID="466ef870ec0f712eccaef4ec9243d8d944e5e34cdfd646447f73219aff2ee84a" exitCode=0 Aug 04 12:32:36.846234 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.846146 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6" Aug 04 12:32:36.846432 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.846148 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6" event={"ID":"f79ddf8e-5d4f-4449-bebc-5cba12c47f4e","Type":"ContainerDied","Data":"466ef870ec0f712eccaef4ec9243d8d944e5e34cdfd646447f73219aff2ee84a"} Aug 04 12:32:36.846558 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.846485 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6" event={"ID":"f79ddf8e-5d4f-4449-bebc-5cba12c47f4e","Type":"ContainerDied","Data":"a7941dbd24f4ec9477e4d9dd157d049bac58c10aafa4d51abb539c261a322dde"} Aug 04 12:32:36.853794 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.853778 2570 scope.go:117] "RemoveContainer" containerID="2e0aeeaac89cd4d521567628f101bed9faef4e73be92dd1aa2abc5e29f125979" Aug 04 12:32:36.854045 ip-10-0-128-47 kubenswrapper[2570]: E0804 12:32:36.854028 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e0aeeaac89cd4d521567628f101bed9faef4e73be92dd1aa2abc5e29f125979\": container with ID starting with 2e0aeeaac89cd4d521567628f101bed9faef4e73be92dd1aa2abc5e29f125979 not found: ID does not exist" containerID="2e0aeeaac89cd4d521567628f101bed9faef4e73be92dd1aa2abc5e29f125979" Aug 04 12:32:36.854111 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.854053 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e0aeeaac89cd4d521567628f101bed9faef4e73be92dd1aa2abc5e29f125979"} err="failed to get container status \"2e0aeeaac89cd4d521567628f101bed9faef4e73be92dd1aa2abc5e29f125979\": rpc error: code = NotFound desc = could not find container \"2e0aeeaac89cd4d521567628f101bed9faef4e73be92dd1aa2abc5e29f125979\": container with ID starting with 2e0aeeaac89cd4d521567628f101bed9faef4e73be92dd1aa2abc5e29f125979 not found: ID does not exist" Aug 04 12:32:36.854111 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.854068 2570 scope.go:117] "RemoveContainer" containerID="466ef870ec0f712eccaef4ec9243d8d944e5e34cdfd646447f73219aff2ee84a" Aug 04 12:32:36.861919 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.861904 2570 scope.go:117] "RemoveContainer" containerID="466ef870ec0f712eccaef4ec9243d8d944e5e34cdfd646447f73219aff2ee84a" Aug 04 12:32:36.862140 ip-10-0-128-47 kubenswrapper[2570]: E0804 12:32:36.862122 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"466ef870ec0f712eccaef4ec9243d8d944e5e34cdfd646447f73219aff2ee84a\": container with ID starting with 466ef870ec0f712eccaef4ec9243d8d944e5e34cdfd646447f73219aff2ee84a not found: ID does not exist" containerID="466ef870ec0f712eccaef4ec9243d8d944e5e34cdfd646447f73219aff2ee84a" Aug 04 12:32:36.862206 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.862147 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"466ef870ec0f712eccaef4ec9243d8d944e5e34cdfd646447f73219aff2ee84a"} err="failed to get container status \"466ef870ec0f712eccaef4ec9243d8d944e5e34cdfd646447f73219aff2ee84a\": rpc error: code = NotFound desc = could not find container \"466ef870ec0f712eccaef4ec9243d8d944e5e34cdfd646447f73219aff2ee84a\": container with ID starting with 466ef870ec0f712eccaef4ec9243d8d944e5e34cdfd646447f73219aff2ee84a not found: ID does not exist" Aug 04 12:32:36.877566 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.877545 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756"] Aug 04 12:32:36.882067 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.882042 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756"] Aug 04 12:32:36.902862 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.902840 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6"] Aug 04 12:32:36.907888 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:36.907867 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6"] Aug 04 12:32:38.195115 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:38.195078 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8" path="/var/lib/kubelet/pods/a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8/volumes" Aug 04 12:32:38.195488 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:38.195418 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f79ddf8e-5d4f-4449-bebc-5cba12c47f4e" path="/var/lib/kubelet/pods/f79ddf8e-5d4f-4449-bebc-5cba12c47f4e/volumes" Aug 04 12:32:47.592088 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:47.592051 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a888a4-fnhkr"] Aug 04 12:32:47.592653 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:47.592634 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f79ddf8e-5d4f-4449-bebc-5cba12c47f4e" containerName="payload-pre-processing" Aug 04 12:32:47.592737 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:47.592656 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f79ddf8e-5d4f-4449-bebc-5cba12c47f4e" containerName="payload-pre-processing" Aug 04 12:32:47.592737 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:47.592672 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8" containerName="payload-processing" Aug 04 12:32:47.592737 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:47.592680 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8" containerName="payload-processing" Aug 04 12:32:47.592737 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:47.592721 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="222451ee-b365-4f39-90f3-8f93e88a733c" containerName="revoke-keys" Aug 04 12:32:47.592737 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:47.592730 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="222451ee-b365-4f39-90f3-8f93e88a733c" containerName="revoke-keys" Aug 04 12:32:47.592980 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:47.592821 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f79ddf8e-5d4f-4449-bebc-5cba12c47f4e" containerName="payload-pre-processing" Aug 04 12:32:47.592980 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:47.592837 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="222451ee-b365-4f39-90f3-8f93e88a733c" containerName="revoke-keys" Aug 04 12:32:47.592980 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:47.592847 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a04255c3-0bc9-4a2b-a81c-6cbd6e460fe8" containerName="payload-processing" Aug 04 12:32:47.597199 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:47.597180 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a888a4-fnhkr" Aug 04 12:32:47.601043 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:47.601023 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pv6cr\"" Aug 04 12:32:47.611373 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:47.611350 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a888a4-fnhkr"] Aug 04 12:32:47.685400 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:47.685363 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb89d08b-a3df-4842-9927-d17411a06246-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-a888a4-fnhkr\" (UID: \"eb89d08b-a3df-4842-9927-d17411a06246\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a888a4-fnhkr" Aug 04 12:32:47.786698 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:47.786666 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb89d08b-a3df-4842-9927-d17411a06246-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-a888a4-fnhkr\" (UID: \"eb89d08b-a3df-4842-9927-d17411a06246\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a888a4-fnhkr" Aug 04 12:32:47.787574 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:47.787551 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb89d08b-a3df-4842-9927-d17411a06246-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-a888a4-fnhkr\" (UID: \"eb89d08b-a3df-4842-9927-d17411a06246\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a888a4-fnhkr" Aug 04 12:32:47.907091 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:47.907013 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a888a4-fnhkr" Aug 04 12:32:48.260336 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:32:48.260303 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb89d08b_a3df_4842_9927_d17411a06246.slice/crio-ddd469f82ccb937886f6e851c99ba1d9b519362d64bf14af8cc20d2e063a3a34 WatchSource:0}: Error finding container ddd469f82ccb937886f6e851c99ba1d9b519362d64bf14af8cc20d2e063a3a34: Status 404 returned error can't find the container with id ddd469f82ccb937886f6e851c99ba1d9b519362d64bf14af8cc20d2e063a3a34 Aug 04 12:32:48.261717 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:48.261693 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a888a4-fnhkr"] Aug 04 12:32:48.907253 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:48.907218 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a888a4-fnhkr" event={"ID":"eb89d08b-a3df-4842-9927-d17411a06246","Type":"ContainerStarted","Data":"962e70884414b79707468cf160676d65765bc89b2b03a20b84eba5859be03b62"} Aug 04 12:32:48.907634 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:48.907302 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a888a4-fnhkr" event={"ID":"eb89d08b-a3df-4842-9927-d17411a06246","Type":"ContainerStarted","Data":"ddd469f82ccb937886f6e851c99ba1d9b519362d64bf14af8cc20d2e063a3a34"} Aug 04 12:32:49.913720 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:49.913685 2570 generic.go:356] "Generic (PLEG): container finished" podID="eb89d08b-a3df-4842-9927-d17411a06246" containerID="962e70884414b79707468cf160676d65765bc89b2b03a20b84eba5859be03b62" exitCode=0 Aug 04 12:32:49.914105 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:49.913770 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a888a4-fnhkr" event={"ID":"eb89d08b-a3df-4842-9927-d17411a06246","Type":"ContainerDied","Data":"962e70884414b79707468cf160676d65765bc89b2b03a20b84eba5859be03b62"} Aug 04 12:32:51.050674 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:51.050652 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a888a4-fnhkr" Aug 04 12:32:51.215878 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:51.215797 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb89d08b-a3df-4842-9927-d17411a06246-maas-api-service-ca\") pod \"eb89d08b-a3df-4842-9927-d17411a06246\" (UID: \"eb89d08b-a3df-4842-9927-d17411a06246\") " Aug 04 12:32:51.216151 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:51.216127 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb89d08b-a3df-4842-9927-d17411a06246-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eb89d08b-a3df-4842-9927-d17411a06246" (UID: "eb89d08b-a3df-4842-9927-d17411a06246"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:32:51.317402 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:51.317366 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb89d08b-a3df-4842-9927-d17411a06246-maas-api-service-ca\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:32:51.925325 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:51.925286 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a888a4-fnhkr" event={"ID":"eb89d08b-a3df-4842-9927-d17411a06246","Type":"ContainerDied","Data":"ddd469f82ccb937886f6e851c99ba1d9b519362d64bf14af8cc20d2e063a3a34"} Aug 04 12:32:51.925325 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:51.925317 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a888a4-fnhkr" Aug 04 12:32:51.925557 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:32:51.925341 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ddd469f82ccb937886f6e851c99ba1d9b519362d64bf14af8cc20d2e063a3a34" Aug 04 12:33:03.083893 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.083808 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a888a4-29764110-cfdpk"] Aug 04 12:33:03.086673 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.086650 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a888a4-29764110-cfdpk"] Aug 04 12:33:03.118230 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.118200 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d"] Aug 04 12:33:03.118516 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.118461 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d" podUID="ddfb6594-e3fd-487a-9556-f1c779926ec0" containerName="payload-processing" containerID="cri-o://4c5bc762f2663390db7a822e8629cd809906fbb2d59e056eec60501072552ee7" gracePeriod=30 Aug 04 12:33:03.125479 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.125430 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h"] Aug 04 12:33:03.126012 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.125898 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h" podUID="fd830dcf-41a3-423d-8e54-6d9438ee0db9" containerName="payload-pre-processing" containerID="cri-o://a1b02f06c6ef534061755aaf70748d55072593ef6d9014651c5f1407ede16336" gracePeriod=30 Aug 04 12:33:03.363206 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.363108 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a888a4-fnhkr"] Aug 04 12:33:03.366205 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.366177 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-a888a4-fnhkr"] Aug 04 12:33:03.411959 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.411934 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d" Aug 04 12:33:03.414533 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.414516 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h" Aug 04 12:33:03.422981 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.422962 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2nmk\" (UniqueName: \"kubernetes.io/projected/ddfb6594-e3fd-487a-9556-f1c779926ec0-kube-api-access-k2nmk\") pod \"ddfb6594-e3fd-487a-9556-f1c779926ec0\" (UID: \"ddfb6594-e3fd-487a-9556-f1c779926ec0\") " Aug 04 12:33:03.423083 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.423000 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddfb6594-e3fd-487a-9556-f1c779926ec0-plugins-config-volume\") pod \"ddfb6594-e3fd-487a-9556-f1c779926ec0\" (UID: \"ddfb6594-e3fd-487a-9556-f1c779926ec0\") " Aug 04 12:33:03.423083 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.423029 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd830dcf-41a3-423d-8e54-6d9438ee0db9-plugins-config-volume\") pod \"fd830dcf-41a3-423d-8e54-6d9438ee0db9\" (UID: \"fd830dcf-41a3-423d-8e54-6d9438ee0db9\") " Aug 04 12:33:03.423203 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.423169 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqlc6\" (UniqueName: \"kubernetes.io/projected/fd830dcf-41a3-423d-8e54-6d9438ee0db9-kube-api-access-bqlc6\") pod \"fd830dcf-41a3-423d-8e54-6d9438ee0db9\" (UID: \"fd830dcf-41a3-423d-8e54-6d9438ee0db9\") " Aug 04 12:33:03.423373 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.423353 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd830dcf-41a3-423d-8e54-6d9438ee0db9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fd830dcf-41a3-423d-8e54-6d9438ee0db9" (UID: "fd830dcf-41a3-423d-8e54-6d9438ee0db9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:33:03.423429 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.423352 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddfb6594-e3fd-487a-9556-f1c779926ec0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ddfb6594-e3fd-487a-9556-f1c779926ec0" (UID: "ddfb6594-e3fd-487a-9556-f1c779926ec0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:33:03.423505 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.423488 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddfb6594-e3fd-487a-9556-f1c779926ec0-plugins-config-volume\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:33:03.423567 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.423506 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd830dcf-41a3-423d-8e54-6d9438ee0db9-plugins-config-volume\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:33:03.424960 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.424941 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddfb6594-e3fd-487a-9556-f1c779926ec0-kube-api-access-k2nmk" (OuterVolumeSpecName: "kube-api-access-k2nmk") pod "ddfb6594-e3fd-487a-9556-f1c779926ec0" (UID: "ddfb6594-e3fd-487a-9556-f1c779926ec0"). InnerVolumeSpecName "kube-api-access-k2nmk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:33:03.425050 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.425031 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd830dcf-41a3-423d-8e54-6d9438ee0db9-kube-api-access-bqlc6" (OuterVolumeSpecName: "kube-api-access-bqlc6") pod "fd830dcf-41a3-423d-8e54-6d9438ee0db9" (UID: "fd830dcf-41a3-423d-8e54-6d9438ee0db9"). InnerVolumeSpecName "kube-api-access-bqlc6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:33:03.524109 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.524072 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2nmk\" (UniqueName: \"kubernetes.io/projected/ddfb6594-e3fd-487a-9556-f1c779926ec0-kube-api-access-k2nmk\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:33:03.524109 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.524098 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqlc6\" (UniqueName: \"kubernetes.io/projected/fd830dcf-41a3-423d-8e54-6d9438ee0db9-kube-api-access-bqlc6\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:33:03.985774 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.985738 2570 generic.go:356] "Generic (PLEG): container finished" podID="ddfb6594-e3fd-487a-9556-f1c779926ec0" containerID="4c5bc762f2663390db7a822e8629cd809906fbb2d59e056eec60501072552ee7" exitCode=0 Aug 04 12:33:03.985980 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.985789 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d" Aug 04 12:33:03.985980 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.985819 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d" event={"ID":"ddfb6594-e3fd-487a-9556-f1c779926ec0","Type":"ContainerDied","Data":"4c5bc762f2663390db7a822e8629cd809906fbb2d59e056eec60501072552ee7"} Aug 04 12:33:03.985980 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.985870 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d" event={"ID":"ddfb6594-e3fd-487a-9556-f1c779926ec0","Type":"ContainerDied","Data":"9b915c58f9e9d45840f577319bd0ab9cc9fc338030ffad460579c5b8e0115874"} Aug 04 12:33:03.985980 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.985894 2570 scope.go:117] "RemoveContainer" containerID="4c5bc762f2663390db7a822e8629cd809906fbb2d59e056eec60501072552ee7" Aug 04 12:33:03.987098 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.987077 2570 generic.go:356] "Generic (PLEG): container finished" podID="fd830dcf-41a3-423d-8e54-6d9438ee0db9" containerID="a1b02f06c6ef534061755aaf70748d55072593ef6d9014651c5f1407ede16336" exitCode=0 Aug 04 12:33:03.987192 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.987111 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h" event={"ID":"fd830dcf-41a3-423d-8e54-6d9438ee0db9","Type":"ContainerDied","Data":"a1b02f06c6ef534061755aaf70748d55072593ef6d9014651c5f1407ede16336"} Aug 04 12:33:03.987192 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.987126 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h" Aug 04 12:33:03.987192 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.987149 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h" event={"ID":"fd830dcf-41a3-423d-8e54-6d9438ee0db9","Type":"ContainerDied","Data":"cde99ac77f9fbcc2167df80afb15cb4423e7cfb4e0708c81b2f8fa82c94e0815"} Aug 04 12:33:03.996586 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.996566 2570 scope.go:117] "RemoveContainer" containerID="4c5bc762f2663390db7a822e8629cd809906fbb2d59e056eec60501072552ee7" Aug 04 12:33:03.998056 ip-10-0-128-47 kubenswrapper[2570]: E0804 12:33:03.998031 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c5bc762f2663390db7a822e8629cd809906fbb2d59e056eec60501072552ee7\": container with ID starting with 4c5bc762f2663390db7a822e8629cd809906fbb2d59e056eec60501072552ee7 not found: ID does not exist" containerID="4c5bc762f2663390db7a822e8629cd809906fbb2d59e056eec60501072552ee7" Aug 04 12:33:03.998152 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.998067 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c5bc762f2663390db7a822e8629cd809906fbb2d59e056eec60501072552ee7"} err="failed to get container status \"4c5bc762f2663390db7a822e8629cd809906fbb2d59e056eec60501072552ee7\": rpc error: code = NotFound desc = could not find container \"4c5bc762f2663390db7a822e8629cd809906fbb2d59e056eec60501072552ee7\": container with ID starting with 4c5bc762f2663390db7a822e8629cd809906fbb2d59e056eec60501072552ee7 not found: ID does not exist" Aug 04 12:33:03.998152 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:03.998089 2570 scope.go:117] "RemoveContainer" containerID="a1b02f06c6ef534061755aaf70748d55072593ef6d9014651c5f1407ede16336" Aug 04 12:33:04.006768 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:04.006750 2570 scope.go:117] "RemoveContainer" containerID="a1b02f06c6ef534061755aaf70748d55072593ef6d9014651c5f1407ede16336" Aug 04 12:33:04.007022 ip-10-0-128-47 kubenswrapper[2570]: E0804 12:33:04.006981 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1b02f06c6ef534061755aaf70748d55072593ef6d9014651c5f1407ede16336\": container with ID starting with a1b02f06c6ef534061755aaf70748d55072593ef6d9014651c5f1407ede16336 not found: ID does not exist" containerID="a1b02f06c6ef534061755aaf70748d55072593ef6d9014651c5f1407ede16336" Aug 04 12:33:04.007022 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:04.007009 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1b02f06c6ef534061755aaf70748d55072593ef6d9014651c5f1407ede16336"} err="failed to get container status \"a1b02f06c6ef534061755aaf70748d55072593ef6d9014651c5f1407ede16336\": rpc error: code = NotFound desc = could not find container \"a1b02f06c6ef534061755aaf70748d55072593ef6d9014651c5f1407ede16336\": container with ID starting with a1b02f06c6ef534061755aaf70748d55072593ef6d9014651c5f1407ede16336 not found: ID does not exist" Aug 04 12:33:04.020780 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:04.020756 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d"] Aug 04 12:33:04.026902 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:04.026885 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d"] Aug 04 12:33:04.040371 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:04.040349 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h"] Aug 04 12:33:04.043229 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:04.043211 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h"] Aug 04 12:33:04.195103 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:04.195078 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92f5134d-346b-416b-b3f5-a6706f816468" path="/var/lib/kubelet/pods/92f5134d-346b-416b-b3f5-a6706f816468/volumes" Aug 04 12:33:04.195421 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:04.195377 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddfb6594-e3fd-487a-9556-f1c779926ec0" path="/var/lib/kubelet/pods/ddfb6594-e3fd-487a-9556-f1c779926ec0/volumes" Aug 04 12:33:04.195706 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:04.195693 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb89d08b-a3df-4842-9927-d17411a06246" path="/var/lib/kubelet/pods/eb89d08b-a3df-4842-9927-d17411a06246/volumes" Aug 04 12:33:04.195999 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:04.195987 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd830dcf-41a3-423d-8e54-6d9438ee0db9" path="/var/lib/kubelet/pods/fd830dcf-41a3-423d-8e54-6d9438ee0db9/volumes" Aug 04 12:33:08.378712 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:08.378683 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 12:33:08.385683 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:08.385662 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-88j8c_2e8c00dc-adfc-4152-901e-ce48741e273c/ovn-acl-logging/0.log" Aug 04 12:33:52.618275 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:52.618237 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0778c1-d92pr"] Aug 04 12:33:52.618797 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:52.618612 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddfb6594-e3fd-487a-9556-f1c779926ec0" containerName="payload-processing" Aug 04 12:33:52.618797 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:52.618623 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddfb6594-e3fd-487a-9556-f1c779926ec0" containerName="payload-processing" Aug 04 12:33:52.618797 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:52.618637 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd830dcf-41a3-423d-8e54-6d9438ee0db9" containerName="payload-pre-processing" Aug 04 12:33:52.618797 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:52.618642 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd830dcf-41a3-423d-8e54-6d9438ee0db9" containerName="payload-pre-processing" Aug 04 12:33:52.618797 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:52.618650 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb89d08b-a3df-4842-9927-d17411a06246" containerName="revoke-keys" Aug 04 12:33:52.618797 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:52.618656 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb89d08b-a3df-4842-9927-d17411a06246" containerName="revoke-keys" Aug 04 12:33:52.618797 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:52.618711 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddfb6594-e3fd-487a-9556-f1c779926ec0" containerName="payload-processing" Aug 04 12:33:52.618797 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:52.618720 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb89d08b-a3df-4842-9927-d17411a06246" containerName="revoke-keys" Aug 04 12:33:52.618797 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:52.618727 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd830dcf-41a3-423d-8e54-6d9438ee0db9" containerName="payload-pre-processing" Aug 04 12:33:52.621410 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:52.621395 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0778c1-d92pr" Aug 04 12:33:52.624155 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:52.624128 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pv6cr\"" Aug 04 12:33:52.630926 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:52.630896 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0778c1-d92pr"] Aug 04 12:33:52.652752 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:52.652720 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c998b0bd-a9b2-4885-a6fd-2b91ca6f0814-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-0778c1-d92pr\" (UID: \"c998b0bd-a9b2-4885-a6fd-2b91ca6f0814\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0778c1-d92pr" Aug 04 12:33:52.754297 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:52.754255 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c998b0bd-a9b2-4885-a6fd-2b91ca6f0814-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-0778c1-d92pr\" (UID: \"c998b0bd-a9b2-4885-a6fd-2b91ca6f0814\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0778c1-d92pr" Aug 04 12:33:52.754914 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:52.754889 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c998b0bd-a9b2-4885-a6fd-2b91ca6f0814-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-0778c1-d92pr\" (UID: \"c998b0bd-a9b2-4885-a6fd-2b91ca6f0814\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0778c1-d92pr" Aug 04 12:33:52.932975 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:52.932883 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0778c1-d92pr" Aug 04 12:33:53.073360 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:53.073332 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0778c1-d92pr"] Aug 04 12:33:53.076716 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:33:53.076682 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc998b0bd_a9b2_4885_a6fd_2b91ca6f0814.slice/crio-a842e5fe93e31dbdd826f9e53e6b5ff6ad969b082ddd047ca0fc430c7c54d1ce WatchSource:0}: Error finding container a842e5fe93e31dbdd826f9e53e6b5ff6ad969b082ddd047ca0fc430c7c54d1ce: Status 404 returned error can't find the container with id a842e5fe93e31dbdd826f9e53e6b5ff6ad969b082ddd047ca0fc430c7c54d1ce Aug 04 12:33:53.225480 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:53.225427 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0778c1-d92pr" event={"ID":"c998b0bd-a9b2-4885-a6fd-2b91ca6f0814","Type":"ContainerStarted","Data":"a842e5fe93e31dbdd826f9e53e6b5ff6ad969b082ddd047ca0fc430c7c54d1ce"} Aug 04 12:33:54.231437 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:54.231400 2570 generic.go:356] "Generic (PLEG): container finished" podID="c998b0bd-a9b2-4885-a6fd-2b91ca6f0814" containerID="ce59b98da9cde2437bc9482d3a999d4cd99377fc2097661fbeb4e0c8c01bc583" exitCode=0 Aug 04 12:33:54.231825 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:54.231481 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0778c1-d92pr" event={"ID":"c998b0bd-a9b2-4885-a6fd-2b91ca6f0814","Type":"ContainerDied","Data":"ce59b98da9cde2437bc9482d3a999d4cd99377fc2097661fbeb4e0c8c01bc583"} Aug 04 12:33:55.375655 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:55.375628 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0778c1-d92pr" Aug 04 12:33:55.478413 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:55.478378 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c998b0bd-a9b2-4885-a6fd-2b91ca6f0814-maas-api-service-ca\") pod \"c998b0bd-a9b2-4885-a6fd-2b91ca6f0814\" (UID: \"c998b0bd-a9b2-4885-a6fd-2b91ca6f0814\") " Aug 04 12:33:55.478779 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:55.478755 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c998b0bd-a9b2-4885-a6fd-2b91ca6f0814-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c998b0bd-a9b2-4885-a6fd-2b91ca6f0814" (UID: "c998b0bd-a9b2-4885-a6fd-2b91ca6f0814"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:33:55.579757 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:55.579677 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c998b0bd-a9b2-4885-a6fd-2b91ca6f0814-maas-api-service-ca\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:33:56.245508 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:56.245477 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0778c1-d92pr" Aug 04 12:33:56.245654 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:56.245483 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0778c1-d92pr" event={"ID":"c998b0bd-a9b2-4885-a6fd-2b91ca6f0814","Type":"ContainerDied","Data":"a842e5fe93e31dbdd826f9e53e6b5ff6ad969b082ddd047ca0fc430c7c54d1ce"} Aug 04 12:33:56.245654 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:33:56.245598 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a842e5fe93e31dbdd826f9e53e6b5ff6ad969b082ddd047ca0fc430c7c54d1ce" Aug 04 12:34:03.067267 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.067219 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf"] Aug 04 12:34:03.067790 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.067462 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf" podUID="ce40eea1-0ec5-4c56-8af4-0a9f4480266f" containerName="payload-processing" containerID="cri-o://922d247a2c271b5f50e8ea4096f08b4fb5efcba4d89e88402039f2710161f53c" gracePeriod=30 Aug 04 12:34:03.073891 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.073856 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb"] Aug 04 12:34:03.074133 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.074088 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb" podUID="808db485-b189-4cbe-8a89-767d553ba7cf" containerName="payload-pre-processing" containerID="cri-o://714fa479b42ef6604a989aada220e3b1e2e00769b926ed4631cf96ee6516a80e" gracePeriod=30 Aug 04 12:34:03.206344 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.206310 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0778c1-d92pr"] Aug 04 12:34:03.217980 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.217948 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0778c1-d92pr"] Aug 04 12:34:03.285993 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.285959 2570 generic.go:356] "Generic (PLEG): container finished" podID="ce40eea1-0ec5-4c56-8af4-0a9f4480266f" containerID="922d247a2c271b5f50e8ea4096f08b4fb5efcba4d89e88402039f2710161f53c" exitCode=0 Aug 04 12:34:03.286158 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.286059 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf" event={"ID":"ce40eea1-0ec5-4c56-8af4-0a9f4480266f","Type":"ContainerDied","Data":"922d247a2c271b5f50e8ea4096f08b4fb5efcba4d89e88402039f2710161f53c"} Aug 04 12:34:03.287883 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.287842 2570 generic.go:356] "Generic (PLEG): container finished" podID="808db485-b189-4cbe-8a89-767d553ba7cf" containerID="714fa479b42ef6604a989aada220e3b1e2e00769b926ed4631cf96ee6516a80e" exitCode=0 Aug 04 12:34:03.287883 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.287868 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb" event={"ID":"808db485-b189-4cbe-8a89-767d553ba7cf","Type":"ContainerDied","Data":"714fa479b42ef6604a989aada220e3b1e2e00769b926ed4631cf96ee6516a80e"} Aug 04 12:34:03.349887 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.349859 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf" Aug 04 12:34:03.352824 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.352804 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb" Aug 04 12:34:03.449463 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.449429 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce40eea1-0ec5-4c56-8af4-0a9f4480266f-plugins-config-volume\") pod \"ce40eea1-0ec5-4c56-8af4-0a9f4480266f\" (UID: \"ce40eea1-0ec5-4c56-8af4-0a9f4480266f\") " Aug 04 12:34:03.449637 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.449493 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/808db485-b189-4cbe-8a89-767d553ba7cf-plugins-config-volume\") pod \"808db485-b189-4cbe-8a89-767d553ba7cf\" (UID: \"808db485-b189-4cbe-8a89-767d553ba7cf\") " Aug 04 12:34:03.449637 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.449514 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2jng\" (UniqueName: \"kubernetes.io/projected/808db485-b189-4cbe-8a89-767d553ba7cf-kube-api-access-c2jng\") pod \"808db485-b189-4cbe-8a89-767d553ba7cf\" (UID: \"808db485-b189-4cbe-8a89-767d553ba7cf\") " Aug 04 12:34:03.449637 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.449560 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mnnr\" (UniqueName: \"kubernetes.io/projected/ce40eea1-0ec5-4c56-8af4-0a9f4480266f-kube-api-access-9mnnr\") pod \"ce40eea1-0ec5-4c56-8af4-0a9f4480266f\" (UID: \"ce40eea1-0ec5-4c56-8af4-0a9f4480266f\") " Aug 04 12:34:03.449910 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.449879 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce40eea1-0ec5-4c56-8af4-0a9f4480266f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ce40eea1-0ec5-4c56-8af4-0a9f4480266f" (UID: "ce40eea1-0ec5-4c56-8af4-0a9f4480266f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:34:03.450021 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.449925 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/808db485-b189-4cbe-8a89-767d553ba7cf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "808db485-b189-4cbe-8a89-767d553ba7cf" (UID: "808db485-b189-4cbe-8a89-767d553ba7cf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:34:03.451697 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.451673 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce40eea1-0ec5-4c56-8af4-0a9f4480266f-kube-api-access-9mnnr" (OuterVolumeSpecName: "kube-api-access-9mnnr") pod "ce40eea1-0ec5-4c56-8af4-0a9f4480266f" (UID: "ce40eea1-0ec5-4c56-8af4-0a9f4480266f"). InnerVolumeSpecName "kube-api-access-9mnnr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:34:03.451790 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.451767 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/808db485-b189-4cbe-8a89-767d553ba7cf-kube-api-access-c2jng" (OuterVolumeSpecName: "kube-api-access-c2jng") pod "808db485-b189-4cbe-8a89-767d553ba7cf" (UID: "808db485-b189-4cbe-8a89-767d553ba7cf"). InnerVolumeSpecName "kube-api-access-c2jng". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:34:03.550495 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.550466 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mnnr\" (UniqueName: \"kubernetes.io/projected/ce40eea1-0ec5-4c56-8af4-0a9f4480266f-kube-api-access-9mnnr\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:34:03.550495 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.550491 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce40eea1-0ec5-4c56-8af4-0a9f4480266f-plugins-config-volume\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:34:03.550495 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.550502 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/808db485-b189-4cbe-8a89-767d553ba7cf-plugins-config-volume\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:34:03.550713 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:03.550511 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2jng\" (UniqueName: \"kubernetes.io/projected/808db485-b189-4cbe-8a89-767d553ba7cf-kube-api-access-c2jng\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:34:04.194759 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:04.194725 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c998b0bd-a9b2-4885-a6fd-2b91ca6f0814" path="/var/lib/kubelet/pods/c998b0bd-a9b2-4885-a6fd-2b91ca6f0814/volumes" Aug 04 12:34:04.294246 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:04.294213 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf" Aug 04 12:34:04.294409 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:04.294213 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf" event={"ID":"ce40eea1-0ec5-4c56-8af4-0a9f4480266f","Type":"ContainerDied","Data":"a08bd8d6ca282892dd07a6a0aedf15e644a91832474085da00ab732c80009942"} Aug 04 12:34:04.294409 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:04.294354 2570 scope.go:117] "RemoveContainer" containerID="922d247a2c271b5f50e8ea4096f08b4fb5efcba4d89e88402039f2710161f53c" Aug 04 12:34:04.295669 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:04.295593 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb" Aug 04 12:34:04.295669 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:04.295624 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb" event={"ID":"808db485-b189-4cbe-8a89-767d553ba7cf","Type":"ContainerDied","Data":"9b79906dbc882177f101b955f3e1e1cc359267c1167c3f6f7cd61aea3ad1bc78"} Aug 04 12:34:04.302922 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:04.302907 2570 scope.go:117] "RemoveContainer" containerID="714fa479b42ef6604a989aada220e3b1e2e00769b926ed4631cf96ee6516a80e" Aug 04 12:34:04.314873 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:04.314849 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb"] Aug 04 12:34:04.323910 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:04.323887 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb"] Aug 04 12:34:04.337153 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:04.337121 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf"] Aug 04 12:34:04.342687 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:04.342666 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf"] Aug 04 12:34:06.195038 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:06.195004 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="808db485-b189-4cbe-8a89-767d553ba7cf" path="/var/lib/kubelet/pods/808db485-b189-4cbe-8a89-767d553ba7cf/volumes" Aug 04 12:34:06.195415 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:06.195350 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce40eea1-0ec5-4c56-8af4-0a9f4480266f" path="/var/lib/kubelet/pods/ce40eea1-0ec5-4c56-8af4-0a9f4480266f/volumes" Aug 04 12:34:10.342461 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:10.342409 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7013e-mfbwz"] Aug 04 12:34:10.342991 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:10.342794 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="808db485-b189-4cbe-8a89-767d553ba7cf" containerName="payload-pre-processing" Aug 04 12:34:10.342991 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:10.342807 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="808db485-b189-4cbe-8a89-767d553ba7cf" containerName="payload-pre-processing" Aug 04 12:34:10.342991 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:10.342820 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce40eea1-0ec5-4c56-8af4-0a9f4480266f" containerName="payload-processing" Aug 04 12:34:10.342991 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:10.342826 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce40eea1-0ec5-4c56-8af4-0a9f4480266f" containerName="payload-processing" Aug 04 12:34:10.342991 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:10.342832 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c998b0bd-a9b2-4885-a6fd-2b91ca6f0814" containerName="revoke-keys" Aug 04 12:34:10.342991 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:10.342839 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c998b0bd-a9b2-4885-a6fd-2b91ca6f0814" containerName="revoke-keys" Aug 04 12:34:10.342991 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:10.342888 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c998b0bd-a9b2-4885-a6fd-2b91ca6f0814" containerName="revoke-keys" Aug 04 12:34:10.342991 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:10.342898 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="808db485-b189-4cbe-8a89-767d553ba7cf" containerName="payload-pre-processing" Aug 04 12:34:10.342991 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:10.342904 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce40eea1-0ec5-4c56-8af4-0a9f4480266f" containerName="payload-processing" Aug 04 12:34:10.347788 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:10.347760 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7013e-mfbwz" Aug 04 12:34:10.350798 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:10.350759 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pv6cr\"" Aug 04 12:34:10.353768 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:10.353744 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7013e-mfbwz"] Aug 04 12:34:10.513830 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:10.513794 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d215d6-db69-4de6-8892-a483e0e670c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-a7013e-mfbwz\" (UID: \"11d215d6-db69-4de6-8892-a483e0e670c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7013e-mfbwz" Aug 04 12:34:10.614866 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:10.614775 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d215d6-db69-4de6-8892-a483e0e670c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-a7013e-mfbwz\" (UID: \"11d215d6-db69-4de6-8892-a483e0e670c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7013e-mfbwz" Aug 04 12:34:10.615411 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:10.615390 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d215d6-db69-4de6-8892-a483e0e670c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-a7013e-mfbwz\" (UID: \"11d215d6-db69-4de6-8892-a483e0e670c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7013e-mfbwz" Aug 04 12:34:10.662196 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:10.662169 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7013e-mfbwz" Aug 04 12:34:10.794865 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:10.793732 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7013e-mfbwz"] Aug 04 12:34:11.331830 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:11.331786 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7013e-mfbwz" event={"ID":"11d215d6-db69-4de6-8892-a483e0e670c5","Type":"ContainerStarted","Data":"a464ecad8461fc03942e2fb096611111e7e94ad94f55be4e30e19a66c55c0713"} Aug 04 12:34:11.331999 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:11.331847 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7013e-mfbwz" event={"ID":"11d215d6-db69-4de6-8892-a483e0e670c5","Type":"ContainerStarted","Data":"bd3f58adea8c4c484d1a01237ea1cea5688118c99334948ce1f7b7a75f292ee4"} Aug 04 12:34:11.362877 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:11.362826 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7013e-mfbwz" podStartSLOduration=1.36281075 podStartE2EDuration="1.36281075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 12:34:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:34:11.361179343 +0000 UTC m=+3663.605898887" watchObservedRunningTime="2026-08-04 12:34:11.36281075 +0000 UTC m=+3663.607530291" Aug 04 12:34:12.337805 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:12.337771 2570 generic.go:356] "Generic (PLEG): container finished" podID="11d215d6-db69-4de6-8892-a483e0e670c5" containerID="a464ecad8461fc03942e2fb096611111e7e94ad94f55be4e30e19a66c55c0713" exitCode=0 Aug 04 12:34:12.337990 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:12.337855 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7013e-mfbwz" event={"ID":"11d215d6-db69-4de6-8892-a483e0e670c5","Type":"ContainerDied","Data":"a464ecad8461fc03942e2fb096611111e7e94ad94f55be4e30e19a66c55c0713"} Aug 04 12:34:13.470798 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:13.470775 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7013e-mfbwz" Aug 04 12:34:13.642379 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:13.642303 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d215d6-db69-4de6-8892-a483e0e670c5-maas-api-service-ca\") pod \"11d215d6-db69-4de6-8892-a483e0e670c5\" (UID: \"11d215d6-db69-4de6-8892-a483e0e670c5\") " Aug 04 12:34:13.642688 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:13.642663 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11d215d6-db69-4de6-8892-a483e0e670c5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "11d215d6-db69-4de6-8892-a483e0e670c5" (UID: "11d215d6-db69-4de6-8892-a483e0e670c5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:34:13.743513 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:13.743480 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d215d6-db69-4de6-8892-a483e0e670c5-maas-api-service-ca\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:34:14.348421 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:14.348390 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7013e-mfbwz" Aug 04 12:34:14.348606 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:14.348397 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7013e-mfbwz" event={"ID":"11d215d6-db69-4de6-8892-a483e0e670c5","Type":"ContainerDied","Data":"bd3f58adea8c4c484d1a01237ea1cea5688118c99334948ce1f7b7a75f292ee4"} Aug 04 12:34:14.348606 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:14.348503 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bd3f58adea8c4c484d1a01237ea1cea5688118c99334948ce1f7b7a75f292ee4" Aug 04 12:34:20.795955 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:20.795922 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7013e-29764110-ngtbf"] Aug 04 12:34:20.802238 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:20.802209 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7013e-29764110-ngtbf"] Aug 04 12:34:20.835201 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:20.835165 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs"] Aug 04 12:34:20.835711 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:20.835409 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs" podUID="a7d6c595-2807-4fca-9dcb-96d66bd9682a" containerName="payload-pre-processing" containerID="cri-o://9cae0ac98d13edfdf8f018767af334e4145570e5bca13d653e347c799444bc2e" gracePeriod=30 Aug 04 12:34:20.975317 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:20.975283 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7013e-mfbwz"] Aug 04 12:34:20.980172 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:20.980145 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7013e-mfbwz"] Aug 04 12:34:21.094745 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:21.094722 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs" Aug 04 12:34:21.212366 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:21.212331 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7d6c595-2807-4fca-9dcb-96d66bd9682a-plugins-config-volume\") pod \"a7d6c595-2807-4fca-9dcb-96d66bd9682a\" (UID: \"a7d6c595-2807-4fca-9dcb-96d66bd9682a\") " Aug 04 12:34:21.212553 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:21.212376 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44vnv\" (UniqueName: \"kubernetes.io/projected/a7d6c595-2807-4fca-9dcb-96d66bd9682a-kube-api-access-44vnv\") pod \"a7d6c595-2807-4fca-9dcb-96d66bd9682a\" (UID: \"a7d6c595-2807-4fca-9dcb-96d66bd9682a\") " Aug 04 12:34:21.212725 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:21.212703 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7d6c595-2807-4fca-9dcb-96d66bd9682a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a7d6c595-2807-4fca-9dcb-96d66bd9682a" (UID: "a7d6c595-2807-4fca-9dcb-96d66bd9682a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:34:21.214431 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:21.214408 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7d6c595-2807-4fca-9dcb-96d66bd9682a-kube-api-access-44vnv" (OuterVolumeSpecName: "kube-api-access-44vnv") pod "a7d6c595-2807-4fca-9dcb-96d66bd9682a" (UID: "a7d6c595-2807-4fca-9dcb-96d66bd9682a"). InnerVolumeSpecName "kube-api-access-44vnv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:34:21.313303 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:21.313263 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7d6c595-2807-4fca-9dcb-96d66bd9682a-plugins-config-volume\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:34:21.313303 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:21.313298 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44vnv\" (UniqueName: \"kubernetes.io/projected/a7d6c595-2807-4fca-9dcb-96d66bd9682a-kube-api-access-44vnv\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:34:21.387656 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:21.387570 2570 generic.go:356] "Generic (PLEG): container finished" podID="a7d6c595-2807-4fca-9dcb-96d66bd9682a" containerID="9cae0ac98d13edfdf8f018767af334e4145570e5bca13d653e347c799444bc2e" exitCode=0 Aug 04 12:34:21.387656 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:21.387611 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs" Aug 04 12:34:21.387656 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:21.387639 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs" event={"ID":"a7d6c595-2807-4fca-9dcb-96d66bd9682a","Type":"ContainerDied","Data":"9cae0ac98d13edfdf8f018767af334e4145570e5bca13d653e347c799444bc2e"} Aug 04 12:34:21.387898 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:21.387681 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs" event={"ID":"a7d6c595-2807-4fca-9dcb-96d66bd9682a","Type":"ContainerDied","Data":"81977c5142f85d73bfccab84f5b9d2f833a4adb2e06b0ed96c4fb359d821d8ba"} Aug 04 12:34:21.387898 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:21.387704 2570 scope.go:117] "RemoveContainer" containerID="9cae0ac98d13edfdf8f018767af334e4145570e5bca13d653e347c799444bc2e" Aug 04 12:34:21.396635 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:21.396617 2570 scope.go:117] "RemoveContainer" containerID="9cae0ac98d13edfdf8f018767af334e4145570e5bca13d653e347c799444bc2e" Aug 04 12:34:21.396884 ip-10-0-128-47 kubenswrapper[2570]: E0804 12:34:21.396867 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cae0ac98d13edfdf8f018767af334e4145570e5bca13d653e347c799444bc2e\": container with ID starting with 9cae0ac98d13edfdf8f018767af334e4145570e5bca13d653e347c799444bc2e not found: ID does not exist" containerID="9cae0ac98d13edfdf8f018767af334e4145570e5bca13d653e347c799444bc2e" Aug 04 12:34:21.396934 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:21.396893 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cae0ac98d13edfdf8f018767af334e4145570e5bca13d653e347c799444bc2e"} err="failed to get container status \"9cae0ac98d13edfdf8f018767af334e4145570e5bca13d653e347c799444bc2e\": rpc error: code = NotFound desc = could not find container \"9cae0ac98d13edfdf8f018767af334e4145570e5bca13d653e347c799444bc2e\": container with ID starting with 9cae0ac98d13edfdf8f018767af334e4145570e5bca13d653e347c799444bc2e not found: ID does not exist" Aug 04 12:34:21.409682 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:21.409655 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs"] Aug 04 12:34:21.416185 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:21.416162 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs"] Aug 04 12:34:22.194758 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:22.194722 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11d215d6-db69-4de6-8892-a483e0e670c5" path="/var/lib/kubelet/pods/11d215d6-db69-4de6-8892-a483e0e670c5/volumes" Aug 04 12:34:22.195126 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:22.195009 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c63924b-cb3b-45e9-826d-614cd9fe81ae" path="/var/lib/kubelet/pods/8c63924b-cb3b-45e9-826d-614cd9fe81ae/volumes" Aug 04 12:34:22.195283 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:22.195270 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7d6c595-2807-4fca-9dcb-96d66bd9682a" path="/var/lib/kubelet/pods/a7d6c595-2807-4fca-9dcb-96d66bd9682a/volumes" Aug 04 12:34:38.472099 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:38.472007 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-038ca6-29764110-4jm9f"] Aug 04 12:34:38.474414 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:38.474392 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-038ca6-29764110-4jm9f"] Aug 04 12:34:38.497876 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:38.497849 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49"] Aug 04 12:34:38.498094 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:38.498053 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49" podUID="2515f6d0-9c47-4c2f-9a99-bdb975c8884c" containerName="payload-processing" containerID="cri-o://58ef048ea7b1c9d2a7f8445b80653e9e4046cdc955f3372287248ae474b4d2df" gracePeriod=30 Aug 04 12:34:38.751711 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:38.751685 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49" Aug 04 12:34:38.868998 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:38.868968 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhpzk\" (UniqueName: \"kubernetes.io/projected/2515f6d0-9c47-4c2f-9a99-bdb975c8884c-kube-api-access-vhpzk\") pod \"2515f6d0-9c47-4c2f-9a99-bdb975c8884c\" (UID: \"2515f6d0-9c47-4c2f-9a99-bdb975c8884c\") " Aug 04 12:34:38.869168 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:38.869064 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2515f6d0-9c47-4c2f-9a99-bdb975c8884c-plugins-config-volume\") pod \"2515f6d0-9c47-4c2f-9a99-bdb975c8884c\" (UID: \"2515f6d0-9c47-4c2f-9a99-bdb975c8884c\") " Aug 04 12:34:38.869461 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:38.869426 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2515f6d0-9c47-4c2f-9a99-bdb975c8884c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2515f6d0-9c47-4c2f-9a99-bdb975c8884c" (UID: "2515f6d0-9c47-4c2f-9a99-bdb975c8884c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 12:34:38.870901 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:38.870877 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2515f6d0-9c47-4c2f-9a99-bdb975c8884c-kube-api-access-vhpzk" (OuterVolumeSpecName: "kube-api-access-vhpzk") pod "2515f6d0-9c47-4c2f-9a99-bdb975c8884c" (UID: "2515f6d0-9c47-4c2f-9a99-bdb975c8884c"). InnerVolumeSpecName "kube-api-access-vhpzk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 12:34:38.969886 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:38.969853 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhpzk\" (UniqueName: \"kubernetes.io/projected/2515f6d0-9c47-4c2f-9a99-bdb975c8884c-kube-api-access-vhpzk\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:34:38.969886 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:38.969884 2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2515f6d0-9c47-4c2f-9a99-bdb975c8884c-plugins-config-volume\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\"" Aug 04 12:34:39.475639 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:39.475606 2570 generic.go:356] "Generic (PLEG): container finished" podID="2515f6d0-9c47-4c2f-9a99-bdb975c8884c" containerID="58ef048ea7b1c9d2a7f8445b80653e9e4046cdc955f3372287248ae474b4d2df" exitCode=0 Aug 04 12:34:39.476043 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:39.475655 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49" Aug 04 12:34:39.476043 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:39.475695 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49" event={"ID":"2515f6d0-9c47-4c2f-9a99-bdb975c8884c","Type":"ContainerDied","Data":"58ef048ea7b1c9d2a7f8445b80653e9e4046cdc955f3372287248ae474b4d2df"} Aug 04 12:34:39.476043 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:39.475749 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49" event={"ID":"2515f6d0-9c47-4c2f-9a99-bdb975c8884c","Type":"ContainerDied","Data":"2f224097bd089d880d6f80985b0f9cc6eb9c582069b55d6db0fee27f34b7136b"} Aug 04 12:34:39.476043 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:39.475779 2570 scope.go:117] "RemoveContainer" containerID="58ef048ea7b1c9d2a7f8445b80653e9e4046cdc955f3372287248ae474b4d2df" Aug 04 12:34:39.484708 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:39.484679 2570 scope.go:117] "RemoveContainer" containerID="58ef048ea7b1c9d2a7f8445b80653e9e4046cdc955f3372287248ae474b4d2df" Aug 04 12:34:39.484954 ip-10-0-128-47 kubenswrapper[2570]: E0804 12:34:39.484934 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58ef048ea7b1c9d2a7f8445b80653e9e4046cdc955f3372287248ae474b4d2df\": container with ID starting with 58ef048ea7b1c9d2a7f8445b80653e9e4046cdc955f3372287248ae474b4d2df not found: ID does not exist" containerID="58ef048ea7b1c9d2a7f8445b80653e9e4046cdc955f3372287248ae474b4d2df" Aug 04 12:34:39.485005 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:39.484963 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58ef048ea7b1c9d2a7f8445b80653e9e4046cdc955f3372287248ae474b4d2df"} err="failed to get container status \"58ef048ea7b1c9d2a7f8445b80653e9e4046cdc955f3372287248ae474b4d2df\": rpc error: code = NotFound desc = could not find container \"58ef048ea7b1c9d2a7f8445b80653e9e4046cdc955f3372287248ae474b4d2df\": container with ID starting with 58ef048ea7b1c9d2a7f8445b80653e9e4046cdc955f3372287248ae474b4d2df not found: ID does not exist" Aug 04 12:34:39.498350 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:39.498324 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49"] Aug 04 12:34:39.504701 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:39.504681 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49"] Aug 04 12:34:40.194423 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:40.194390 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07da324e-3515-4528-bf85-29930ce23f9b" path="/var/lib/kubelet/pods/07da324e-3515-4528-bf85-29930ce23f9b/volumes" Aug 04 12:34:40.194757 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:40.194742 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2515f6d0-9c47-4c2f-9a99-bdb975c8884c" path="/var/lib/kubelet/pods/2515f6d0-9c47-4c2f-9a99-bdb975c8884c/volumes" Aug 04 12:34:52.528203 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:52.528175 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f8dd78b9-nhds8_6de73521-bfa3-4490-b98d-104a8cfbc93c/copy-manifests/0.log" Aug 04 12:34:52.535941 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:52.535890 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f8dd78b9-nhds8_6de73521-bfa3-4490-b98d-104a8cfbc93c/manager/0.log" Aug 04 12:34:53.414908 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:53.414869 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29764110-wzpv8_a8b25435-ae90-48e1-949d-fce97991cf99/cleanup/0.log" Aug 04 12:34:53.523364 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:53.523339 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-jnxg5_494f1c12-d6ed-4720-8e7d-d33e5a21fe28/postgres/0.log" Aug 04 12:34:54.719400 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:54.719372 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-7skwx_f6453805-e250-4db3-9998-c2ae5888ee09/manager/0.log" Aug 04 12:34:55.378929 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:55.378904 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-kdhht_0741d66e-850c-4c18-a29a-3834ad2fd527/discovery/0.log" Aug 04 12:34:56.592619 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:56.592589 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-hc8jf_b2e4de77-e5dd-4e29-9138-7d79bdeebae8/main/0.log" Aug 04 12:34:56.697709 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:34:56.697680 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7gzkv_1e26593c-4b41-45c9-964f-af9d699521ab/main/0.log" Aug 04 12:35:05.054844 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:05.054814 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lcqpb_e3158842-36a1-4213-af82-c6db89481715/global-pull-secret-syncer/0.log" Aug 04 12:35:05.143875 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:05.143845 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4wnrp_9dafc945-e2f2-4b29-8b6d-b27765cc5e6e/konnectivity-agent/0.log" Aug 04 12:35:05.206714 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:05.206687 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-47.ec2.internal_b3c19a22d5a3d4dd837bfebae044979e/haproxy/0.log" Aug 04 12:35:10.266911 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:10.266880 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-7skwx_f6453805-e250-4db3-9998-c2ae5888ee09/manager/0.log" Aug 04 12:35:12.433872 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:12.433774 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-567c6fcf9-2wgtk_692fda94-e203-446b-97cb-4a43752ac992/metrics-server/0.log" Aug 04 12:35:12.469642 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:12.469611 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-6dffbbd67-kdbbp_c7c99760-43cf-4240-b1bb-713b92892169/monitoring-plugin/0.log" Aug 04 12:35:12.743130 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:12.743103 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bfqvz_fb8c73dd-6a50-4085-99ad-9ed552b4f3f3/node-exporter/0.log" Aug 04 12:35:12.770573 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:12.770547 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bfqvz_fb8c73dd-6a50-4085-99ad-9ed552b4f3f3/kube-rbac-proxy/0.log" Aug 04 12:35:12.800464 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:12.800421 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bfqvz_fb8c73dd-6a50-4085-99ad-9ed552b4f3f3/init-textfile/0.log" Aug 04 12:35:12.937813 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:12.937781 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_00c9278b-a606-4fd1-9f15-6d43483c0088/prometheus/0.log" Aug 04 12:35:12.968858 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:12.968831 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_00c9278b-a606-4fd1-9f15-6d43483c0088/config-reloader/0.log" Aug 04 12:35:13.001338 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:13.001258 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_00c9278b-a606-4fd1-9f15-6d43483c0088/thanos-sidecar/0.log" Aug 04 12:35:13.028031 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:13.028010 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_00c9278b-a606-4fd1-9f15-6d43483c0088/kube-rbac-proxy-web/0.log" Aug 04 12:35:13.055566 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:13.055528 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_00c9278b-a606-4fd1-9f15-6d43483c0088/kube-rbac-proxy/0.log" Aug 04 12:35:13.087023 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:13.087002 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_00c9278b-a606-4fd1-9f15-6d43483c0088/kube-rbac-proxy-thanos/0.log" Aug 04 12:35:13.116664 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:13.116640 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_00c9278b-a606-4fd1-9f15-6d43483c0088/init-config-reloader/0.log" Aug 04 12:35:13.153269 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:13.153236 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-5xshl_f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a/prometheus-operator/0.log" Aug 04 12:35:13.178005 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:13.177981 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-5xshl_f99581d4-f9ea-4f60-a6ce-a8f1fd03c14a/kube-rbac-proxy/0.log" Aug 04 12:35:13.209178 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:13.209152 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-66685ff6d4-9459q_775c79ad-948e-4fde-9707-7ebb75bb9981/prometheus-operator-admission-webhook/0.log" Aug 04 12:35:13.427668 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:13.427591 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54d87cb5fc-k4tvk_a278c772-e331-4f77-989e-ac533e463d21/thanos-query/0.log" Aug 04 12:35:13.452544 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:13.452515 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54d87cb5fc-k4tvk_a278c772-e331-4f77-989e-ac533e463d21/kube-rbac-proxy-web/0.log" Aug 04 12:35:13.478069 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:13.478031 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54d87cb5fc-k4tvk_a278c772-e331-4f77-989e-ac533e463d21/kube-rbac-proxy/0.log" Aug 04 12:35:13.506245 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:13.506220 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54d87cb5fc-k4tvk_a278c772-e331-4f77-989e-ac533e463d21/prom-label-proxy/0.log" Aug 04 12:35:13.537393 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:13.537357 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54d87cb5fc-k4tvk_a278c772-e331-4f77-989e-ac533e463d21/kube-rbac-proxy-rules/0.log" Aug 04 12:35:13.569423 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:13.569398 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-54d87cb5fc-k4tvk_a278c772-e331-4f77-989e-ac533e463d21/kube-rbac-proxy-metrics/0.log" Aug 04 12:35:14.915112 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:14.915076 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7"] Aug 04 12:35:14.915576 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:14.915392 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11d215d6-db69-4de6-8892-a483e0e670c5" containerName="revoke-keys" Aug 04 12:35:14.915576 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:14.915402 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="11d215d6-db69-4de6-8892-a483e0e670c5" containerName="revoke-keys" Aug 04 12:35:14.915576 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:14.915423 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7d6c595-2807-4fca-9dcb-96d66bd9682a" containerName="payload-pre-processing" Aug 04 12:35:14.915576 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:14.915429 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7d6c595-2807-4fca-9dcb-96d66bd9682a" containerName="payload-pre-processing" Aug 04 12:35:14.915576 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:14.915437 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2515f6d0-9c47-4c2f-9a99-bdb975c8884c" containerName="payload-processing" Aug 04 12:35:14.915576 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:14.915470 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2515f6d0-9c47-4c2f-9a99-bdb975c8884c" containerName="payload-processing" Aug 04 12:35:14.915576 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:14.915543 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7d6c595-2807-4fca-9dcb-96d66bd9682a" containerName="payload-pre-processing" Aug 04 12:35:14.915576 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:14.915551 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="11d215d6-db69-4de6-8892-a483e0e670c5" containerName="revoke-keys" Aug 04 12:35:14.915576 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:14.915557 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2515f6d0-9c47-4c2f-9a99-bdb975c8884c" containerName="payload-processing" Aug 04 12:35:14.919835 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:14.919818 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7" Aug 04 12:35:14.923786 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:14.923765 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8l5dn\"/\"default-dockercfg-b5kh6\"" Aug 04 12:35:14.925048 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:14.925030 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8l5dn\"/\"kube-root-ca.crt\"" Aug 04 12:35:14.925128 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:14.925033 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8l5dn\"/\"openshift-service-ca.crt\"" Aug 04 12:35:14.935412 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:14.935388 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7"] Aug 04 12:35:14.982638 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:14.982602 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cf8ee008-72c2-4bfc-8e18-f2ae931ba815-podres\") pod \"perf-node-gather-daemonset-c48d7\" (UID: \"cf8ee008-72c2-4bfc-8e18-f2ae931ba815\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7" Aug 04 12:35:14.982803 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:14.982649 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkrp7\" (UniqueName: \"kubernetes.io/projected/cf8ee008-72c2-4bfc-8e18-f2ae931ba815-kube-api-access-lkrp7\") pod \"perf-node-gather-daemonset-c48d7\" (UID: \"cf8ee008-72c2-4bfc-8e18-f2ae931ba815\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7" Aug 04 12:35:14.982803 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:14.982729 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cf8ee008-72c2-4bfc-8e18-f2ae931ba815-proc\") pod \"perf-node-gather-daemonset-c48d7\" (UID: \"cf8ee008-72c2-4bfc-8e18-f2ae931ba815\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7" Aug 04 12:35:14.982803 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:14.982785 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf8ee008-72c2-4bfc-8e18-f2ae931ba815-sys\") pod \"perf-node-gather-daemonset-c48d7\" (UID: \"cf8ee008-72c2-4bfc-8e18-f2ae931ba815\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7" Aug 04 12:35:14.982912 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:14.982827 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cf8ee008-72c2-4bfc-8e18-f2ae931ba815-lib-modules\") pod \"perf-node-gather-daemonset-c48d7\" (UID: \"cf8ee008-72c2-4bfc-8e18-f2ae931ba815\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7" Aug 04 12:35:15.084269 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:15.084230 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf8ee008-72c2-4bfc-8e18-f2ae931ba815-sys\") pod \"perf-node-gather-daemonset-c48d7\" (UID: \"cf8ee008-72c2-4bfc-8e18-f2ae931ba815\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7" Aug 04 12:35:15.084269 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:15.084277 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cf8ee008-72c2-4bfc-8e18-f2ae931ba815-lib-modules\") pod \"perf-node-gather-daemonset-c48d7\" (UID: \"cf8ee008-72c2-4bfc-8e18-f2ae931ba815\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7" Aug 04 12:35:15.084545 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:15.084303 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cf8ee008-72c2-4bfc-8e18-f2ae931ba815-podres\") pod \"perf-node-gather-daemonset-c48d7\" (UID: \"cf8ee008-72c2-4bfc-8e18-f2ae931ba815\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7" Aug 04 12:35:15.084545 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:15.084334 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkrp7\" (UniqueName: \"kubernetes.io/projected/cf8ee008-72c2-4bfc-8e18-f2ae931ba815-kube-api-access-lkrp7\") pod \"perf-node-gather-daemonset-c48d7\" (UID: \"cf8ee008-72c2-4bfc-8e18-f2ae931ba815\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7" Aug 04 12:35:15.084545 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:15.084368 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cf8ee008-72c2-4bfc-8e18-f2ae931ba815-proc\") pod \"perf-node-gather-daemonset-c48d7\" (UID: \"cf8ee008-72c2-4bfc-8e18-f2ae931ba815\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7" Aug 04 12:35:15.084545 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:15.084381 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf8ee008-72c2-4bfc-8e18-f2ae931ba815-sys\") pod \"perf-node-gather-daemonset-c48d7\" (UID: \"cf8ee008-72c2-4bfc-8e18-f2ae931ba815\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7" Aug 04 12:35:15.084545 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:15.084419 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cf8ee008-72c2-4bfc-8e18-f2ae931ba815-lib-modules\") pod \"perf-node-gather-daemonset-c48d7\" (UID: \"cf8ee008-72c2-4bfc-8e18-f2ae931ba815\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7" Aug 04 12:35:15.084545 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:15.084469 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cf8ee008-72c2-4bfc-8e18-f2ae931ba815-proc\") pod \"perf-node-gather-daemonset-c48d7\" (UID: \"cf8ee008-72c2-4bfc-8e18-f2ae931ba815\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7" Aug 04 12:35:15.084545 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:15.084501 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cf8ee008-72c2-4bfc-8e18-f2ae931ba815-podres\") pod \"perf-node-gather-daemonset-c48d7\" (UID: \"cf8ee008-72c2-4bfc-8e18-f2ae931ba815\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7" Aug 04 12:35:15.095843 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:15.095819 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkrp7\" (UniqueName: \"kubernetes.io/projected/cf8ee008-72c2-4bfc-8e18-f2ae931ba815-kube-api-access-lkrp7\") pod \"perf-node-gather-daemonset-c48d7\" (UID: \"cf8ee008-72c2-4bfc-8e18-f2ae931ba815\") " pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7" Aug 04 12:35:15.229675 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:15.229645 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7" Aug 04 12:35:15.360197 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:15.360169 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7"] Aug 04 12:35:15.362886 ip-10-0-128-47 kubenswrapper[2570]: W0804 12:35:15.362851 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcf8ee008_72c2_4bfc_8e18_f2ae931ba815.slice/crio-8db013785d7310f8347fc4135a0698eb2c5bce12604ae0ef5dae21d693a66ade WatchSource:0}: Error finding container 8db013785d7310f8347fc4135a0698eb2c5bce12604ae0ef5dae21d693a66ade: Status 404 returned error can't find the container with id 8db013785d7310f8347fc4135a0698eb2c5bce12604ae0ef5dae21d693a66ade Aug 04 12:35:15.658352 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:15.658255 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7" event={"ID":"cf8ee008-72c2-4bfc-8e18-f2ae931ba815","Type":"ContainerStarted","Data":"dabb6c6b1bc2541ac9f3add5278907983ffc2324a59216e829afaaccb1af78d8"} Aug 04 12:35:15.658352 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:15.658314 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7" event={"ID":"cf8ee008-72c2-4bfc-8e18-f2ae931ba815","Type":"ContainerStarted","Data":"8db013785d7310f8347fc4135a0698eb2c5bce12604ae0ef5dae21d693a66ade"} Aug 04 12:35:15.658626 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:15.658384 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7" Aug 04 12:35:15.677849 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:15.677804 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7" podStartSLOduration=1.677791181 podStartE2EDuration="1.677791181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 12:35:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 12:35:15.676647912 +0000 UTC m=+3727.921367453" watchObservedRunningTime="2026-08-04 12:35:15.677791181 +0000 UTC m=+3727.922510722" Aug 04 12:35:17.718524 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:17.718479 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pjxbb_84257250-96bb-4afa-91fa-f2f031eaa074/dns/0.log" Aug 04 12:35:17.739644 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:17.739618 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pjxbb_84257250-96bb-4afa-91fa-f2f031eaa074/kube-rbac-proxy/0.log" Aug 04 12:35:17.832865 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:17.832835 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hbfpg_f8dbbbc0-29a4-4849-9700-f9f8ebb310d8/dns-node-resolver/0.log" Aug 04 12:35:18.396019 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:18.395989 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-789fcb8b4c-mqrqd_f2fef570-6051-45ae-9317-592c2708fa0f/registry/0.log" Aug 04 12:35:18.417786 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:18.417756 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5z7cf_8f2acb16-9405-4d91-853d-6a54e7a03a59/node-ca/0.log" Aug 04 12:35:19.578194 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:19.578164 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-kdhht_0741d66e-850c-4c18-a29a-3834ad2fd527/discovery/0.log" Aug 04 12:35:20.609547 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:20.609519 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-db4rq_7e7bd05e-d544-4ac5-b099-612ea158fc36/serve-healthcheck-canary/0.log" Aug 04 12:35:21.237728 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:21.237702 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-klkv6_2e4855b1-d090-4d03-a7bc-a5fff928a162/kube-rbac-proxy/0.log" Aug 04 12:35:21.260030 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:21.260006 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-klkv6_2e4855b1-d090-4d03-a7bc-a5fff928a162/exporter/0.log" Aug 04 12:35:21.285169 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:21.285137 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-klkv6_2e4855b1-d090-4d03-a7bc-a5fff928a162/extractor/0.log" Aug 04 12:35:21.672355 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:21.672289 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8l5dn/perf-node-gather-daemonset-c48d7" Aug 04 12:35:24.242281 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:24.242251 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f8dd78b9-nhds8_6de73521-bfa3-4490-b98d-104a8cfbc93c/manager/0.log" Aug 04 12:35:24.267397 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:24.267372 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f8dd78b9-nhds8_6de73521-bfa3-4490-b98d-104a8cfbc93c/copy-manifests/0.log" Aug 04 12:35:25.777560 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:25.777529 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-kdhht_0741d66e-850c-4c18-a29a-3834ad2fd527/discovery/0.log" Aug 04 12:35:25.824968 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:25.824944 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-d5b557dbd-gqt96_19ef2037-15c8-4873-b1f7-7bcd7beef87a/manager/0.log" Aug 04 12:35:27.552505 ip-10-0-128-47 kubenswrapper[2570]: I0804 12:35:27.552473 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-kdhht_0741d66e-850c-4c18-a29a-3834ad2fd527/discovery/0.log"